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SUMMARY 

The  mathematical  theory  of  differential  games  is  used  to  study  the 
structure  of  optimal  allocation  strategies  for  some  time-sequential 
combat  games  with  combat  described  by  Lanchester-type  equations  of  warfare. 
Although  most  of  this  work  concerns  the  application  of  existing  theory  for 
the  determination  of  optimal  campaign  strategies  in  various  Lanchester- 
type  differential  games  of  tactical  interest,  theoretical  developments  are 
also  given  consideration. 

Previous  research  by  the  author  had  uncovered  an  important  gap  in  the 
existing  theory  of  differential  games:   no  adequate  theory  of  state  variable 
inequality  constraints  existed  for  differential  games.   Such  a  theory  is 
essential  for  Lanchester-type  differential  games  because,  for  example,  of 
the  requirement  that  force  levels  be  non-negative.   Consequently,  in  this 
report  first-order  necessary  conditions  of  optimality  are  developed  for 
differential  games  with  state  variable  inequality  constraints.   These  results 
are  then  applied  to  the  study  of  optimal  compaign  strategies  in  several 
differential  games  of  tactical  interest. 

Optimal  air-war  campaign  strategies  are  studied  in  a  generalization 
of  a  well-known  tactical  air-war  model.  Previous  work  had  never  given 
adequate  consideration  to  necessary  conditions  of  optimality  with  respect 
to  the  non-negativity  of  aircraft  force  levels.  The  work  at  hand  extends 
previous  work  by  considering  a  value  for  survivors  at  the  end  of  the  air 
war  and  variable  (effectiveness)  coefficients.  Several  specific  problems 
are  solved.  The  effects  of  temporal  variations  in  the  return  from  ground 
support  on  optimal  campaign  strategies  are  studied. 


The  differential  game  model  discussed  above  has  been  criticized 
because  it  does  not  evaluate  air-war  tactics  within  the  context  of  ground- 
war  objectives.   Consequently,  this  model  produces  suboptimization.   Hence, 
a  model  is  considered  which  is  considerably  broader  in  scope,  does  consider 
land-war  objectives,  but  (unfortunately)  is  considerably  less  susceptible 
to  closed-form  analytic  solution.   For  this  new  model  partial  results 
concerning  the  determination  of  optimal  campaign  strategies  are  obtained 
and  contrasted  with  those  for  the  model  previously  discussed  above. 

The  determination  of  optimal  fire  distribution  strategies  for 
supporting  weapon  systems  is  a  major  problem  of  military  operations 
research.   The  author  has  studied  some  aspects  of  this  problem  in  previous 
research.   Motivated  by  a  paper  which  recently  appeared  in  the  open  liter- 
ature, optimal  fire-support  strategies  are  studied  in  an  attack  scenario. 
The  dependence  of  optimal  fire-support  strategies  on  combatant  objectives 
is  examined  for  a  problem  previously  considered  in  the  open  literature: 
necessary  and  sufficient  conditions  on  the  functional  form  of  the  criterion 
functional  (terminal  payoff  only)  for  the  optimal  fire-support  strategies 
to  be  independent  of  force  levels  (at  least  for  a  certain  range  of  values) 
are  developed.   This  shows  that  certain  results  previously  reported  in  the 
literature  are,  in  general,  not  true.   A  variation  of  this  problem  is 
then  considered.   In  contrast  to  the  previous  work  reported  in  the  litera- 
ture, the  attrition  structure  of  the  problem  at  hand  leads  to  the  optimal 
fire-support  strategy  of  the  attacker  requiring  him  to  sometimes  split  his 
artillery  fire  between  enemy  infantry  and  artillery  (counterbattery  fire) . 
Numerical  examples  are  given. 
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Finally,  some  problems  for  possible  future  study  are  discussed. 

The  formulation  of  these  problems  is  based  on  the  author's  past  research 

experiences.   As  is  usually  the  case,  work  on  the  problems  considered 

here  and  elsewhere  has  uncovered  other  aspects  of  interest  that  appear 
to  merit  further  examination. 
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1 •   Introduction . 

This  report  is  the  third  in  a  series  of  reports  which  document 
the  author's  research  on  the  structure  of  optimal  (time-sequential) 
allocation  policies  for  tactical  situations  described  by  Lanchester- 
type  equations  of  warfare.   By  considering  several  combat  scenarios, 
further  insights  have  been  gained  into  such  important  questions  as: 

(1)  How  should  fire  be  distributed  over  targets? 

(2)  How  should  targets  be  selected? 

(3)  Do  target  priorities  change  over  time? 

(4)  Do  force  levels  affect  the  optimal  allocation  strategies? 

(5)  How  does  the  number  of  target  types  affect  the  optimal 
allocation  strategies? 

(6)  Do  conflict  termination  circumstances  affect  the  optimal 
allocation  strategies? 

(7)  How  are  the  optimal  fire  distribution/target  selection 
strategies  affected  by  the  nature  of  the  attrition  processes? 

(8)  What  is  the  effect  of  logistics  constraints  on  such  policies? 

(9)  How  does  the  uncertainty  and  confusion  of  combat  affect 
optimal  allocation  strategies? 

Specific  objectives  of  this  research  have  been  the  identification 
of  decision  parameters  and  the  further  development  of  general  principles 
for  target  selection,  regulation  of  firing  rate,  and  allocation  of 
military  resources  in  dynamic  combat  situations.   A  further  discussion 
of  research  objectives  and  their  relationship  to  defense  planning 
problems  can  be  found  in  the  author's  past  reports*  [11],  [12], 


* 

A  comprehensive  review  of  pertinent  literature  prior  to  1973  in  the 

fields  of  (a)  Lanchester-type  models  of  warfare,  (b)  differential  games, 
and  (c)  optimal  control  of  Lanchester-type  attrition  processes  is  also 
to  be  found  in  [12] . 


It  should  be  pointed  out  that  although  the  quantitative  analysis  of  tactics 
may  be  considered  to  be  a  relatively  new  field  of  military  operations 
research,  there  is  wide-spread  interest  in  such  work  (see  [6],  [8], [9], 
[10], [19]). 

Our  approach  has  been  to  combine  Lanchester-type  models  of  warfare 
with  generalized  control  theory  (both  deterministic  and  stochastic  optimal 
control,  dynamic  programming,  and  differential  games)  [12], [15]  (see  also 
[16]).   This  has  been  done  by  considering  a  sequence  of  concrete  problems. 
Additionally,  the  report  at  hand  contains  some  basic  research  on  the 
theory  of  differential  games.   Past  research  [12], [15]  had  identified  as 
an  important  gap  in  the  existing  theory  of  differential  games  the  lack 
of  adequate  theory  (i.e.  necessary  conditions  of  optimality)  for  problems 
with  state  variable  inequality  constraints  (SVIC's). 

The  organization  of  this  report  is  to  discuss  results  in  general 
terms  in  the  main  body  and  to  leave  details  for  the  appendices.   This 
has  been  partially  necessitated  by  the  wide  scope  of  problems  examined 
(from  theoretical  to  applied) . 

2  .   Guided  Tour  of  t  he  J\pp  end  ices  . 

In  this  section  we  summarize  the  work  which  is  contained  in  the 
appendices  and  explain  why  this  work  was  done.   The  results  reported 
here  may  be  considered  to  be  extensions  of  our  earlier  work  [11], [12], 

[15].   Moreover,  the  work  at  hand  lays  the  foundations  for  more  extensive 
work  on  the  quantitative  analysis  of  tactics  and  applications  of  generalized 

control  theory  to  problems  of  military  operations  research. 


In  Appendix  A  we  develop  a  theory  of  state  variable  inequality 
constraints  for  Lanchester-type  differential  games.   First  order 
necessary  conditions  of  optimality  are  developed  for  the  class  of  problems 
in  which  individual  state  variables  are  bounded  by  functions  of  time. 
Although  this  certainly  is  not  the  most  general  type  of  state  variable 
Inequality  constraint  (SVIC) ,  it  does  include  all  Lanchester-type 
differential  games  that  occur  in  military  operations  research.   First 
order  necessary  conditions  of  optimality  are  also  stated  (but  not  proved) 
for  more  general  problems.   In  particular,  results  are  given  without  proof 
for  problems  with  a  fairly  general  type  of  SVIC.   The  proofs  of  these 
results  will  be  given  in  the  future. 

Recently,  we  pointed  out  [12], [15]  that  no  adequate  multiplier 
conditions  existed  for  (Lanchester-type)  differential  games  with  state 
variable  inequality  constraints  and  gave  this  top  research  priority. 
It  is  indeed  remarkable  that  essentially  all  previous  applications  of 
the  theory  of  differential  games  to  Lanchester-type  problems  (see,  for 
example,  [1] , [ 2] , [ A] , [5 ] , [ 7] )  are  inadequate  in  one  respect  or  another 
with  respect  to  SVIC's.   A  further  discussion  and  substantiation  of 
this  claim  is  to  be  found  in  Appendix  A.   This  work  establishes  the 
mathematical  foundations  for  future  applications  of  the  theory  of  dif- 
ferential games  because  force  levels  must  be,  for  example,  non-negative 
(see  [17]  for  a  discussion  within  the  context  of  Lanchester-type  optimal 
control  problems). 

In  Appendices  B  and  C  we  apply  the  theoretical  results  of  Appendix 
A  to  the  study  of  optimal  air-war  strategies.   This  has  been  over  the 
years  a  problem  of  lasting  interest  for  defense  planners.   In  Appendix  B 


we  consider  a  generalization  of  the  tactical  air-war  game  of   A.  Mengel 
[41  (see  also  [3])  in  which  aircraft  effectiveness  may  vary  over  time  and 
a  residual  value  for  aircraft  at  the  end  of  the  campaign  is  considered. 
Optimal  strategies  are  characterized  for  this  general  problem.   A 
complete  solution  (i.e.  explicit  determination  of  optimal  closed-loop 
strategies)  is  given  for  cases  of  constant  coefficients.   These  results 
are  much  more  general  than  previous  ones  and  provide  much  insight  into 
tradeoffs  between  various  planning  parameters.   Preliminary  results  are 
also  obtained  for  problems  in  which  aircraft  effectiveness  in  the  air 
war  is  constant  but  the  value  of  ground  support  changes  over  time.   The 
latter  is  an  attempt  to  reflect  the  influence  of  combat  air  support  on 
the  ground  war  and  hence  avoid  suboptimization.   Two  cases  of  time- 
dependent  "returns"  from  ground  support  are  considered: 

(a)  linearly-decreasing-with-time  returns  from  ground  support, 

(b)  exponentially-decreasing-with-time  returns  from  ground  support. 
One  criticism  that  has  been  made  of  the  model  considered  in 

Appendix  B  is  that  it  does  not  evaluate  air-war  strategies  within  the 
context  of  ground-war  objectives.  In  other  words,  this  model  (unfor- 
tunately) produces  suboptimization.  Thus,  in  Appendix  C  we  present 
preliminary  results  for  a  model  which  considers  the  development  of 
optimal  air-war  strategies  within  the  context  of  land-war  objectives. 
These  preliminary  results  show  that  the  outcome  of  the  ground  war  may 
be  a  significant  factor  in  the  determination  of  optimal  air-war  strategies, 

The  research  reported  in  Appendices  B  and  C  was  undertaken  because 
an  important  question  for  defense  planners  is  what  are  appropriate 
missions  over  the  course  of  a  campaign  for  tactical  air  power.   The 
answer  to  this  question  has  far-reaching  implications  for  Navy  air  forces 


(both  carrier-based  and  land-based)  and,  of  course,  the  Air  Force. 
Recently,  the  USAF  Studies  and  Analysis  Group  has  been  using  quantitative 
methodology  [19]  in  trying  to  answer  such  questions.   By  considering  the 
results  given  in  Appendices  B  and  C  one  can  begin  to  see  the  effects  of 
the  nature  of  the  combat  optimization  problem  on  the  structure  of  optimal 
air-war  strategies. 

The  determination  of  optimal  fire  distribution  strategies  for 
supporting  weapon  systems  is  a  major  problem  of  military  operations 
research.   This  problem  is  particularly  of  interest  to  the  military 
tactician  so  that  he  may  have  a  clearer  understanding  of  the  circum- 
stances, for  example,  under  which  enemy  infantry  should  be  engaged  by  a 
supporting  weapon  system  (such  as  artillery)  and  those  under  which  "counter- 
battery"  fire  is  to  be  preferred.   Furthermore,  the  study  of  such  tactical 
time-sequential  allocation  problems  is  relevant  to  the  Navy  mission  of 
fire  support  (both  by  ship  gunfire  and  by  carrier-based  air). 

In  Appendix  D  we  further  consider  the  determination  of  optimal  fire 
distribution  strategies  for  supporting  weapon  systems.   In  this  case  it 
is  appropriate  to  call  the  supporting  weapon  system  artillery.   We  consider 
extensions  of  some  recent  work  by  Kawara  [5]  concerning  optimal  strategies 
for  supporting  weapon  systems  in  an  attack  scenario  which  is  a  variation 
of  a  model  considered  by  Weiss  [20].   We  first  examine  for  what  class  of 
criterion  functionals  (terminal  payoff  only)  the  optimal  fire-support 
strategies  are  independent  of  the  force  levels  and  then  examine  the  depen- 
dence  of  the  optimal  fire-support  strategies  upon  the  form  of  the  combat 
attrition  model  by  considering  slightly  different  combat  dynamics  than 
those  considered  by  Kawara  [5]:   we  assume  that  the  attacker's  artillery 


produces  "linear-law"  attrition  (see  [13],  [16])  against  both  the  defender's 
artillery  as  well  as  infantry.   The  development  of  a  complete  solution 
to  the  latter  problem  has  involved  solution  phenomena  not  previously 
encountered  in  a  Lanchester-type  differential  game:   the  dual  variables 
are  discontinuous  across  a  manifold  of  discontinuity  of  both  players' 
strategies . 

The  research  reported  in  Appendix  D  was  undertaken  to  obtain  a 
better  understanding  of  the  dependence  of  the  structure  of  optimal 
fire-support  strategies  upon  model  form.   Previous  work  by  Weiss  [20] 
(see  also  [4])  and  Kawara  [5]  had  given  the  impression  that  an  optimal 
fire-support  strategy  consists  in  always  concentrating  all  fire  on  one 
enemy  target  type.   From  our  previous  work  [13]  on  a  one-sided 
Lanchester-type,  time-sequential  fire  distribution  problem  in  which  each 
of  two  enemy  target  types  undergoes  a  "linear-law"  attrition  process,  we 
knew  that  an  optimal  fire  distribution  policy  could  consist  in  splitting 
one's  fire  between  available  target  types.   This  was  our  motivation  for 
the  examination  of  other  attrition  structures  in  Kawara 's  problem. 
Additionally,  Kawara's  recent  work  [5]  stimulated  our  research  on  the 
influence  of  combatant  objectives  on  optimal  fire-support  strategies 
because  of  Kawara's  remarkable  result  that  (at  least  within  a  certain 
range  of  force  levels)  optimal  fire-support  strategies  do  not  depend  on 
force  levels.   We  were  able  to  show  that  Kawara  had  used  the  only  type  of 
objective  function  that  yeilds  this  result.   Thus,  in  general,  the  result 
is  not  true. 

In  Appendix  E  we  briefly  describe  several  important  problems  for 
possible  future  research.   These  problems  were  formulated  giving 


consideration  to  our  past  research  experiences  (both  that  reported  here 
and  also  elsewhere).   These  problems  may  be  referred  to  as 

(a)  optimal  fire  support  for  several  ground  units, 

(b)  a  Lanchester-type  optimal  control  problem  with  logistics 
considerations , 

(c)  an  examination  of  the  influence  of  the  form  of  the  criterion 
functional  on  optimal  strategies  in  time-sequential  combat 
optimization  problems. 

As  explained  in  Appendix  E,  work  on  these  campaign  analysis  problems 
would  appreciably  enhance  our  understanding  of  methodology  for  the 
optimization  of  combat  dynamics. 
3 .   Summary  of  Research  Findings. 

Here  we  summarize  our  research  results.   We  have  (at  least  partially) 
accomplished  the  following  tasks  which  were  suggested  for  future  research 
in  our  previous  report  [12]:   (a),  (b2),  (e) ,  (g)*,  (il),  (j2),  and  (k) . 
Results  are  organized  under  the  following  headings: 

(1)  solution  methodology  for  time-sequential  combat  games, 

(2)  insights  gained  into  the  optimization  of  combat  dynamics, 

(3)  implications  for  defense  planning. 

Items  (2)  and  (3)  differ  in  that  the  latter  is  a  management-oriented 
digest  of  the  practical  implications  of  our  research,  whereas  the  former 
is  oriented  towards  a  technical  audience.   Further  amplification  of 
results  and  conclusions  is  to  be  found  in  the  appendices. 

a.   Solution  Methodology  for  Time-Sequential  Combat  Games. 

Our  research  has  produced  the  following  results  on  solution 
methodology  for  time-sequential  combat  games.   Specifically,  we  have 
accomplished  the  following: 

*See  [18].  7 


(1)  developed  theory  of  state  variable  inequality  constraints 
for  Lanchester-type  differential  games  (This  theory  is 
essential  for  all  problems  with,  for  example,  negativity 
restrictions  on  force  levels  . ) , 

(2)  demonstrated  use  of  theory  of  state  variable  inequality 
constraints  for  solving  time-sequential  combat  games  by 
developing  solutions  to  several  specific  combat  optimiza- 
tion problems, 

(3)  developed  complete  solution  to  a  fire  support  differential 
game.   This  involved  the  following  technical  difficulties: 

(a)  singular  surfaces, 

(b)  discontinuity  in  the  dual  variables, 

(4)  developed  methodology  for  determining  the  functional  form 
of  the  criterion  functional  (terminal  payoff  here)  which 
yields  optimal  strategies  that  do  not  depend  on  force 
levels  (at  least  for  a  certain  range  of  force  levels), 

(5)  concluded  that  computational  methods  must  give  consideration 
to  structural  properties  of  optimal  strategies  in  idealized 
versions  like  those  considered  in  this  report. 

b .   Insights  Gained  into  the  Optimization  of  Combat  Dynamics . 

Based  on  our  study  of  the  optimization  of  combat  dynamics  using 

the  mathematical  theory  of  differential  games,  we  have  reached  the  following 

conclusions : 

(1)  the  structure  of  optimal  strategies  depends  on  the  following 
factors : 

(a)  criterion  functional, 

(b)  combat  attrition  model, 

(c)  battle  termination  model; 

the  dependence  is  complex,  and  future  research  should  concen- 
trate upon  the  examination  of  numerous  structures  of 
tactical  interest, 

(2)  optimal  fire-support  strategies  depend  on  the  nature  of  the 
attrition  process  produced  by  the  supporting  weapon  system; 
when  the  supporting  weapon  system  produces  casualties  at  a 
rate  proportional  to  the  product  of  the  numbers  of  firers 
and  targets  of  a  particular  type  (see  Appendix  D  for 
mathematical  formulation),  the  optimal  fire-support  strategy 
of  the  supporting  weapon  system  has  the  following  character- 
istics: 


(a)  it  depends  on  the  optimal  strategy  for  the  enemy's 
supporting  weapon  system, 

(b)  the  optimal  distribution  of  supporting  fires  may  be 
(i)  to  concentrate  on  enemy  infantry, 

(ii)  to  split  fire  to  avoid  overkilling, 
(iii)  to  concentrate  on  enemy  artillery, 

(c)  counterbattery  fire  is  the  optimal  strategy  during 
the  early  stages  of  an  attack  and  destruction  of 
enemy  infantry  is  the  optimal  strategy  during  the 
final  moments  (unless  the  ratio  of  defender  infantry 
to  defender  artillery  is  "extremely"  large,  in  which 
case  the  infantry  is  always  engaged) , 

(d)  a  split  of  supporting  fires  between  enemy  infantry 
and  artillery  as  an  optimal  strategy  can  only  occur 
when  enemy  infantry  have  some  effectiveness  against 
friendly  infantry, 

(3)  optimal  air-war  strategies  are  different  for  the  case 
in  which  ground  war  objectives  are  considered  than  for 
that  in  which  they  are  not, 

(4)  optimal  air-war  strategies  (see  Appendix  B  for  scenario) 
depend  on  the  following  factors: 

(a)  residual  value  of  surviving  aircraft, 

(b)  force  levels  (especially  when  a  side  is  going  to 
lose  the  fight  for  air  supremacy:   faced  with  a 
future  loss  of  airpower,  one  may  abandon  the  fight 
for  air  supremacy  and  get  what  ground  support  one 
can  from  his  planes  before  he  loses  them  all), 

(5)  cases  with  variations  over  time  in  weapon  system 
effectiveness  are  best  studied  as  extensions  of  constant 
coefficient  cases. 

c.   Implications  for  Defense  Planning. 

In  our  research  reported  here  we  have  studied  idealizations 

of  allocation  structures  that  commonly  occur  in  defense  planning  studies. 

After  studying  these  idealizations  in  order  to  gain  insight  into  the 

structure  of  optimal  strategies  in  the  complex  real-world  problem,  we 

have  reached  the  following  conclusions  as  to  considerations  that  should 

be  brought  to  the  attention  of  defense  planners.   These  results  should 

be  kept  in  mind  by  practitioners  who  perform  more  detailed  computer 

simulation  studies. 

9 


(1)  The  combat  optimization  problem  should  be  thought  of  as 
consisting  of  three  parts: 

(i)  combatant  objectives, 
(ii)  conflict  termination  conditions, 
(iii)  combat  dynamics. 
Optimal  combat  strategies  depend  on  all  three  of  the 
above.   More  basic  scientific  research  should  be  done  on 
all  three,  especially  the  first  two. 

(2)  Force  levels  do  affect  (either  directly  or  indirectly) 
optimal  combat  strategies. 

(3)  It  may  be  quite  dangerous  to  generalize  optimal  combat 
strategies  developed  for  specific  problems.   At  present, 
more  research  is  needed  on  specific  problems  in  order  to 
develop  an  understanding  of  the  qualifications  that  may 
be  necessary  to  make  about  specific  study  results. 

(4)  Optimal  air-war  strategies  must  be  based  on  ground-war 
objectives.   Suboptimization  results  when  this  is  not 
done.   This  suboptimization  is  a  serious  problem,   since 

it  may  lead  to  winning  the  air  campaign  but  losing  the  war, 

4 .   Suggested  Future  Research  Tasks. 

After  performing  the  research  documented  in  this  report,  we  feel  that 
the  current  state-of-the-art  for  applying  the  theory  of  differential  games 
to  time-sequential  combat  games  is  such  that  more  significant  results 
may  be  readily  obtained  in  the  future.   Moreover,  this  previous  research 
provides  valuable  perspective  for  identifying  what  appear  to  be  the  most 
important  research  tasks  to  be  considered  next.   In  our  opinion  the  most 
important  task  is  to  examine  (and  then  understand)  the  influence  of  objec- 
tives on  optimal  strategies.   Another  important  task  is  to  study  the 
structure  of  optimal  fire-support  strategies. 

Based  on  our  past  research  experience  we  feel  that  there  is  much  to  be 

accomplished  in  the  future.   Specifically,  we  suggest  the  following  as 

future  research  tasks: 

(a)  Examination  of  the  effects  of  campaign  objectives  (as  reflected 
by  the  criterion  functional)  on  the  structure  of  optimal  campaign 
strategies.   In  all  our  past  research  we  have  with  one  exception 
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always  considered  a  linear  utility  for  survivors  in  the  criterion 
functional.   It  is  of  interest  to  examine  how  the  valuation  of 
survivors  affects  the  structure  of  optimal  strategies.   Other 
criterion  functionals  that  might  be  considered  are 

(1)  nonlinear  valuation  of  survivors  (either  the  difference 
or  ratio  of  functions  which  are,  for  example,  concave, 
quasi-concave,  etc.), 

(2)  the  value  of  losses  rather  than  survivors. 

It  would  seem  appropriate  to  begin  such  an  investigation  by  con- 
sidering the  simplest  problem  possible. 

(b)  Further  study  of  optimal  fire-support  strategies.   The  actual 
military  operations  must  be  analyzed  and  decisions  identified. 
For  various  cases,  the  appropriate  scenario  and,  consequently, 
Lanchester-type  model  would  be  developed  (see  Appendix  E) ,  and 
the  optimization  problem  studied.   It  is  felt  that  at  this  time 
it  is  most  important  to  study  various  specific  problems  in  order 

to  gain  insight  into  the  structure  of  optimal  fire-support  strategies. 

(c)  Examination  of  the  effects  of  logistics  constraints  on  optimal 
campaign  strategies.   Models  would  be  developed  to  relate  logistics 
capability  to  combat  effectiveness  and  then  appropriate  combat 
optimization  problems  formulated.   Such  research  would  provide 
insight  into  the  worth  of  the  Navy  logistics  (pipeline)  role  in 
combat  service  support  missions  (see  Appendix  E) . 

(d)  Further  study  the  determination  of  optimal  air-war  strategies 
within  the  context  of  land-war  objectives.   Based  on  preliminary 
results  given  in  this  report,  it  appears  as  though  the  outcome  of 
the  ground  war  is  a  significant  factor*  in  the  determination  of 
optimal  air-war  strategies  and  that  optimal  strategies  developed 
for  a  model  not  considering  land-war  objectives  need  not  be  optimal 
for  a  model  which  does  consider  them.   This  proposed  research 
would  extend  the  results  given  in  Appendix  C  of  the  report  at  hand. 

A  more  comprehensive  (although  somewhat  dated  now)  discussion  of 

suggested  future  research  tasks  can  also  be  found  on  pages  65-67  of  our 

previous  report  [12].   Work  still  remains  to  be  done  on  the  following 

tasks  suggested  there:   (b),  (c),  (d) ,  (f),  (h),  and  (j). 


*This  is  not  considered,  for  example,  in  TAC  CONTENDER  [19] 
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APPENDIX  A:   Necessary  Conditions  of  Optimality  for  Differential  Games 
with  State  Variable  Inequality  Constraints. 

1.   Introduction. 

In  our  most  recent  report  on  applications  of  differential  games  to 
problems  of  military  conflict  [22]  we  suggested  as  a  future  research  task 
that  a  theory  of  state  variable  inequality  constraints  (SVIC's)  be  developed 
for  Lanchester-type  differential  games  (see  [23]).   This  suggestion  was 
based  on  our  discovery  of  an  important  gap  in  the  existing  theory  of 
differential  games:   no  adequate  treatment  of  multiplier  conditions  for 
differential  games  with  SVIC's  then  existed  in  the  literature.   As  we  have 
pointed  out  previously  [23],  [25],  state  variable  inequality  constraints 
are  present  in  all  Lanchester-type  dynamic  tactical  allocation  problems 
because  force  levels  (which  are  represented  by  state  variables)  are  required 
to  be  non-negative  (or  some  equivalent  condition).   Thus,  the  development 
of  a  theory  of  SVIC's  is  essential  for  an  adequate  characterization  of  an 
optimal  strategy  in  such  a  problem  (see  examples  given  in  Appendix  B) .   It 
is  indeed  remarkable  that  this  important  gap  in  theoretical  results  has 
not  even  been  noted  by  other  workers  in  operations  research. 

The  origins  of  the  theory  of  differential  games  are  given  in  [10]. 
Since  the  publication  of  this  paper  in  1965  and  that  of  the  highly  signifi- 
cant book  by  Isaacs  [11] ,  numerous  papers  have  been  published  on  the  subject 
of  differential  games.   Further  references  can  be  found  in  [22].   Necessary 
conditions  of  optimality  have  been  developed  for  differential  games  by 
Isaacs  [11]  and  Berkovitz  [3],  [4].   In  two  highly  technical  and  significant 
papers  [3],  [4],  Berkovitz  developed  necessary  conditions  for  problems 
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subject  to  various  types  of  inequality  constraints  except  those  only 
involving  functions  of  the  state  variables  (i.e.  except  SVIC's).   Varia- 
tional methods  were  used  in  [3]  by  reducing  the  problem  to  two  associated 
variational  problems.   In  [4]  the  properties  of  the  value  function  were 
utilized  to  develop  these  conditions  by  use  of  Isaacs'  "tenet  of  transi- 
tion."  Schmitendorf  and  Citron  [16]  extended  Berkovitz's  variational 
approach  to  second  order  conditions  and  developed  a  conjugate  point  condi- 
tion for  a  class  of  differential  games.   Later,  Schmitendorf  [15]  used  the 
same  approach  to  develop  (both  first  and  also  second  order)  necessary 
conditions  of  optimality  for  (zero-sum  deterministic)  differential  games 
in  which  one  or  both  players  are  restricted  to  use  open-loop  control.   The 
development  of  necessary  conditions  has  also  been  disucssed  by  Chattopadhyay 
[7]. 

In  this  appendix  we  will  develop  necessary  conditions  of  optimality 
for  differential  games  with  SVIC's  by  an  extension  of  the  approach  of 
Berkovitz  [3].   The  development  of  necessary  conditions  for  such  problems 
has  apparently  not  been  accomplished  before  although  sufficient  conditions 
for  optimality  have  been  derived  by  a  number  of  workers  [13],  [18]. 

Quite  independently  of  the  above  developments  A.  Friedman  began  a 
careful  study  of  the  existence  of  value  for  a  differential  game  and  existence 
of  a  saddle  point  in  pure  strategies.   This  work  led  to  the  book  [9]  and 
has  continued.   Differential  games  with  SVIC's  have  been  considered  by 
Friedman,  but  he  has  focused  primarily  on  the  question  of  existence  of 
value  for  them  (see  Chapter  6  of  [9]).   We  have  used  Friedman's  results 
in  our  past  work  [19],  [20],  [21].   However,  Friedman  [9]  does  not  develop 
the  two-sided  analogues  of  well-known  control  theory  multiplier  conditions 
for  constrained  subarcs  which  lie  on  the  boundary  of  the  state  space  [12] . 
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In  fact,  the  treatment  of  the  example  on  pp.  239-240  of  [9]  is  inadequate, 
since  the  approach  fails  to  yield  optimal  strategies  for  minor  modifications 
in  the  problem's  formulation  (see  Appendix  B) .   Moreover,  this  problem  is 
very  important  for  military  operations  research  (see  Appendix  B) . 

The  research  reported  in  this  appendix  is  of  a  more  basic  nature 
than  that  reported  elsewhere  in  this  report.  These  results  are  also  of 
much  wider  application  than  to  the  problems  of  defense  planning  upon  which 
the  author  has  concentrated.  Moreover,  the  derivation  of  these  results 
allows  one  to  identify  the  class  of  problems  to  which  such  multiplier 
conditions  apply.  It  should  be  noted  that  not  all  system  dynamics  give 
rise  to  such  conditions. 

2 .   A  Lanchester-Type  Differential  Game. 

In  [23]  we  coined  the  term  Lanchester-type  differential  game  as 
referring  to  a  differential  game  in  which  the  system  dynamics  are  described 
by  Lanchester-type  equations  of  warfare.   To  be  precise,  we  are  considering 
two-person  zero-sum  deterministic  differential  games  in  which  each  player 
uses  a  closed-loop  (or  feedback)  strategy.   A  formulation  which  is  rele- 
vant to  our  research  on  tactical  allocation  problems  is  as  follows: 

n  m 

maximize  minimize  {  £  w.y.(tf)  -  £  v.x.(tf)}, 

¥        $      i=l    1       i=l  X   1 
ij       ij 

stopping  rule:     tf  -  T  =  0, 


dx.         n 

subject  to:    - —  =  r.  -   )   i|>..a..y.    for   i  =  1, 
bat  dynamics)  j=l   J   J  J 


,m, 


(combat   dynamics) 

r-^  =s.    -      )      <f> .  .  b  .  .  x  .        fo 

[t  1        j=i      XJ    1J    J 


dy .  m 

dl 


i  =   l,...,n,  (1) 
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with 


x.,y.  ^  0   for  all   i,j,    (State  Variable  Inequality  Constraints) 


m 


£  ty . .    £  1   for  j  =  l,...,n,  (Strategic 

Variable 


1=1   1J 


n  Inequality 

V   d>..  £  1   for   j  =  l,...,m,  ■      '     '    \ 

f;  Yij  Constraints) 


where 

x.(t),  y.(t)   are  force  levels, 

r.,  s.   are  replacement  rates, 
11 

v.,  w.   are  the  utilities  assigned  survivors, 
11 

a..   is  the  rate  at  which  one  Y.   unit  can  destroy  X., 
ij  J  i 

b..   is  the  rate  at  which  one  X.   unit  can  destroy  Y., 

<b .  .   is  the  fraction  of  X.   who  fire  at  Y., 

and      il> .  .   is  the  fraction  of  Y.   who  fire  at  X.. 
ij  J  i 

The  above  problem  (1)  contains  all  the  essential  features  (and  can  be 
made  identical  to  by  minor  changes  in  formulation)  of  all  the  Lanchester- 
type  differential  games  studied  previously  in  the  operations  research 
literature  (see  [23]). 

It  should  be  noted  that  we  use  capital  letters  to  denote  a  strategy 
(see  [3]  or  [9]  for  precise  definitions),  whereas  the  corresponding  lower 
case  letter  is  used  to  denote  the  realization  of  the  strategy  (Friedman 
uses  the  term  outcome  (see  pp.  22-23  of  [9]).   A  strategy  is  a  mapping 
from  the  state  space  to  the  space  of  admissible  decisions.   Heuristically , 
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it  is  a  contingincy  plan  that  tells  us  what  to  do  depending  upon  where  we 
find  ourselves  in  the  state  space  (time  may  sometimes  be  an  additional 
state  variable).   In  other  words,  we  have  f  =  ¥(t,x,y)   where  ¥   denotes 

r*~>  ***>  SSS  r^ 

a   q  vector,    x   denotes  an  m  vector,  and   y  denotes  an  n  vector. 

Then  strategic  variable  is  the  realization  (or  outcome)  of  a  strategy  as 
follows : 

♦  ijCt)  =  ^.(t.x.y).  (2) 

As  mentioned  above,  we  will  refer  to  (1)  as  a  Lanchester-type 
differential  game.   This  suggests  that  we  consider  the  following  problem 
(denoted  as  "Problem  I")  for  the  development  of  necessary  conditions  of 
optimality  for  a  class  of  differential  games  with  SVIC's. 


Problem  I.  „ 

T 


maximize  minimize  G(T ,x(T) ,y (T) )  + 
U        V 


L(t,x,y,u,v)dt  , 


0 


subject  to:        x  =  f(t,x,y,v), 

/^     r*s  ^w  <v 

u(t) ,  v(t)   are  unrestricted  (scalar)  strategic  variables, 


x  (t)  >   a(t)   for  all   t  €  [0,T]  (scalar  state 

m 

variable  inequality 
y  (t)  <:  3(t)   for  all   t  €  [0,T]  constraints), 


with  scalar  terminal  condition  F(T,x(T) ,y(T) )  =  0, 


* 


We  use  standard  notation  for  vectors,  etc.  as  defined,  for  example,  on 
pp.  4-6  of  [8]  . 
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where  we  assume  that  all  functions  are  smooth  enough  to  insure  the 
existence  of  all  partial  derivatives  required  in  the  following  analysis. 
In  Problem  I  above,   x   is  an  m-vector  of  state  variables,  and   y   is  an 
n-vector  of  state  variables. 

3.   Mathematical  Preliminaries. 

In  this  section  we  further  establish  our  notation  and  state  our 
assumptions.   We  consider  closed-loop  (or  feedback)  strategies ,  denoted 
as   U(t,x,y)   and  V(t,x,y),   for  the  players.   A  strategy  then  is  the 
specification  of  what  value  should  be  given  to  a  player's  strategic  variable 
at  each  point  in  time  depending  upon  the  time  and  current  values  of  the 
state  variables.   The  realization  (or  outcome)  of  a  strategy  will  be 
denoted  by  the  corresponding  lower  case  letter.   Thus 

u(t)  =  U(t,x,y)   and  v(t)  =  V(t ,x,y) .  (3) 

We  call  u(t)   and  v(t)   strategic  variables. 

The  criterion  functional  is  denoted  by  J,   and  thus  we  have  for 

Problem  I 

T 
f 
J  =  G(T,x(T),y(T))  +   L(t ,x(t) ,y (t) ,u(t) ,v(t) )dt .         (4) 


r**s  **** 


r**  *** 


0 
Player  I  chooses  U(t,x,y)   and  tries  to  maximize  J,   while  Player  II 

chooses  V(t,x,y)   and  tries  to  minimize  J.   We  say  that   (U*,V*)   is  a 

saddle  point  of   J   if 

J(u,V*(t,x,y))  <;  J(U*(t,x,y),V*(t,x,y))  <;  J  (U*(t  ,x,y)  ,v)  .      (SPC) 
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Then   (U*,V*)   is  the  solution  to  Problem  I.   We  denote  the  value  of  the 
differential  game  as  W.   Then 

W  =  J(U*,V*).  (5) 

We  assume  that  Problem  I  has  value  and  a  saddle  point  in  pure  strategies. 
It  will  be  convenient  to  denote  the  vector  of  state  variables  as 

T     T   T 


*"W      *"W 


We  will  then  sometimes  write  terms  in  a  more  convenient  form  as 

L(t,z,u,v)  =  L(t,x,y,u,v) .  (7) 

Although  not  essential  for  the  development  of  necessary  conditions  of 
optimality,  to  insure  the  existence  of  value  for  the  differential  game  we 
assume 

L(t,z,u,v)  =  L  (t,x,y,u)  +  L  (t,x,y,v).  (8) 

It  is  assumed  that  both  players  have  perfect  information  as  to  the 

state  variables.   In  other  words,  both  players  know  precisely  the  current 

state  of  the  system. 

We  assume  that  the  first  time  derivative  of  y  (t)   explicitly 

n 

• 

contains  the  strategic  variable  u  and  that   (y  )   =  g   (t,x,y,u)  ^  0 

n      n    ~  ~ 
u     u 

along  an  optimal  trajectory.   We  further  assume  that  the  first  time 

derivative  of  x  (t)   explicitly  contains  the  strategic  variable  v  and 

that   (x  )   =  f   (t,x,y,v)  ^  0  along  an  optimal  trajectory.   In  other 
m      m    #w  A/ 
v     v 


* 
Several  other  technical  conditions  must  be  satisfied  to  guarantee  the 

existence  of  value  and  of  a  saddle  point  in  pure  strategies  to  Problem  I 

(see  Chapter  6  of  [9]). 
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words,  we  assume  that   x  (t)  ^  a(t)   and   y  (t)  £  B(t)   are  the  appropriate 
types  of  first  order  SVIC's  (see  [12]). 

4 .   Statement  of  Necessary  Conditions  of  Optimality  for  Problem  I. 
First  we  define 

T 
H(t,z,p,q,n,u,v)  =  L(t,z,u,v)  +  p  f(t,x,y,v) 

+   qTg(t,x,y,u)  -  n-.(t){x  -a(t)}  -  n„(t){y  -(B(t)},    (9) 
*<r  l     m  z     n 


is*  is* 


where 


=  0   for  x   >  a(t) 


m 


n1(t) 


^  0   for   x   =  a(t) , 


m 


and 


n2(t) 


0  for  y  <  3(t), 
n 


;>  0  for  y  =  (3(t)  , 
n 


and 


<D(T,z(T),a)  =  G(T,x(T),y(T))  +  cF(T,x(T)  ,y(T)  )  .  (10) 

*s*  is*  is*  rs^  is* 

Let   x*(t)   for   0  £  y  £   T   denote  the  optimal  path  which  results  from 
(U*(t,x,y),  V*(t,x,y)).   We  assume  that  the  solution  to  Problem  I  is  normal 
(see  [1])  and  that  the  optimal  path  is  not  tangent  to  the  terminal  manifold 
or  to  any  manifold  of  discontinuity  of  U*   or  V*  or  to  the  boundary  of 
the  state  space  at  entry  to  a  constrained  subarc  which  lies  on  the  boundary, 
Then,  in  order  that  the  strategy  pair   (U*,V*)   be  a  saddle  point  of  the 
criterion  functional  it  is  necessary  that  there  exist  unique  functions 


*  th 

In  optimal  control  theory  one  says  [6]  that  a  problem  has  a  p —  order 

SVIC  when  the  pHL  time  derivative  of  the  state-variable  constraint  is 

the  first  to  explicitly  contain  the  control  variable. 
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p(t),    q(t) ,    n,  (t) ,      and      n~(t)      and   constants     a,      v    ,      and      v        such 


r+*  rs* 


that  the  following  conditions  hold 


3H    .        3L    T  3&   _ 

-r—  =0  or   -^~  +  q     t—  =  0t 

3u  du    ~   du 

9H    n         9L     T  9~    n 

—  =  0  or   —  +  p  t—  =  0, 

3v  dv    ^   dV 


(11) 

(12) 


•T   _    3L    T  31    T  3^     ,.m.T 

■p   =H   =  —  +  p   -r—  +  q H,  (A  )  , 

~      X    3x    ~   3x    ~   3x      1  ~ 


(13) 


3L  .   T  31  .   T  3&     ,An.T 


I       dL 


3I 


(14) 


T 
P  (T) 


3$ 


3x(T) 


m  T  /"0  for  xm(T)  >  o(T)  , 

-  v  (A  )   where  v1  <  ^   (15) 

1  ~  X  [    :>  0   for  x  (T)  =  ct(T), 


3.  (T)  =  3^TT  "  V2(i  }    Where  V2 


1 


=  0   for  y  (T)  <  (3(T), 


;>  0  for  yn(T)  =  6(T), 


(16) 


where 


H(T)  +  H  +  V]La(T)  +  v2B(T)  =  0, 


(17) 


AJ  = 


6li 
62j 


^   jj  ; 


and   6  .  .  ■ 


1  for  i  =  j, 


0  otherwise. 


These  results  hold  only  when  the  boundary  of  the  state  space  (at  least  the 
part  for  which  x  =  a(t))  is  non-absorbing  (see  [24]).   For  an  absorbing 
state  boundary  we  nave 


=  0  for  x  (T)  >  a(T)  , 


I 

1  \  m 

y  unrestricted   for  x 


^  0   for  x  (T)  =  o(T)   but  x  (t)  >  a(t)   for   t  <  T, 


m 


(T)  =  o(T)   and  x  (t)  =  a(t)   for 
m 

t   <;  t  <:  T  with   t   <  T. 
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Analogous  to  the  Weierstrass  condition,  we  also  have  the  max-min  principle 

H(u,v*)  <:  H(u*,v*)  <:H(u*,v),  (18) 

which  holds  for  all  admissible   u  and   v.   H,  P^(t), 

and   q(t)   are  continuous  (vector-valued)  functions  except  at  manifolds 
of  discontinuity  of  both  U*   and  V*,   where  for  all  differentials   dt, 
dx ,   and   dy  along  the  manifold 

(H+-H")dt  -  (p+-p")Tdx  -  (q+-q~)Tdy  =  0.  (19) 

In  other  words,  at  both  entrances  to  and  exits  from  constrained 
subarcs  on  the  boundary  of  the  state  space,  we  have 

£(t")  =  £(t+),  (20) 


and 


a(t-)  =£(tt}'  (21) 


H(t.)  =  H(t*),  (22) 


except  possibly  at  manifolds  of  discontinuity  of  both  U*   and  V*,   where 

all  the  dual  variables  are  continuous  except  those  corresponding  to   x 

m 

or   y   which  describes  the  boundary, 
n 

The  above  first  order  necessary  conditions  of  optimality  for  Problem 
I_  are  analogous  to  the  optimal  control  theory  results  of  Jacobson,  Lele, 
and  Speyer  [12].   The  above  results  may  also  be  written  in  a  form  analogous 
to  that  of  optimal  control  theory  results  of  Berkovitz  [2]  and  Gamkrelidze 
(see  Chapter  VI  in  [14]).   This  is  done  by  introducing  new  multipliers 
X(t)   such  that 

p(t)  =  \ At)    -   u,(t)Am,  (23) 

~        *vL  1    ~ 

q(t)  =  X    (t)  -  u9(t)An.  (24) 
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It  is  well  known  (see  [24])  that  we  may  then  make  the  following  identifications 

n.(t)  =  -y.(t)   for  i  =  1,2.  (25) 

5 .   Development  of  Necessary  Conditions  of  Optimality. 

It  is  convenient  to  re-write  Problem  I  as  (DG) . 

T 

maximize  minimize  {G(T ,x(T) ,y (T) )  +   L(t ,x,y ,u,v)dt} , 
U       V        ~    ~      J 

subject  to:        x  =  f(t,x,y,v),  (DG) 

^■W       *N*       <■»»*  rs* 

y  ■  g(t,x,y,u) , 

SW       /K*       *S*  ^^/ 

u(t) ,  v(t)   are  unrestricted  (scalar)  strategic  variables, 
x  (t)  ^  a(t)   for  all   t  €  [0,T]      (scalar  state  variable 


m 


y  (t)  <:  g(t)   for  all  t  €  [0,T] 


inequality  constraints) , 


with  scalar  terminal  condition   F(T ,x(T) ,y (T) )  =  0. 

Considering  the  saddle  point  condition  (SPC)  for  the  criterion  functional, 
necessary  conditions  of  optimality  for  (DG)  will  be  developed  according 
to  the  approach  of  Berkovitz  [3]  in  which  maximum  and  minimum  problems 
associated  with  the  game  are  considered.   Necessary  conditions  of  optimality 
are  developed  for  each  of  these  two  problems.   It  is  then  shown  that  these 
two  sets  of  conditions  may  be  written  as  a  single  set  of  necessary  condi- 
tions for  (DG)  . 

We  extend  Berkovitz 's  variational  approach  to  differential  games 
with  SVIC's  by  treating  boundary  segments  by  two  different  methods  in  the 
associated  optimal  control  problems.   For  the  simple  case  of  bounded  state 
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variables  at  hand,  we  may  think  of  both  methods  of  treating  an  SVIC  as 
techniques  in  which  the  dimensionality  of  the  state  space  is  reduced  for 
a  constrained  subarc  which  lies  on  the  boundary  for  a  finite  interval  of 
time. 

5.1.   Necessary  Conditions  for  Control  Problem  A. 

Thus,  we  are  led  to  maximum  and  minimum  problems  associated  with 
(SPC) .   The  associated  maximum  problem  we  denote  as  Control  Problem  A, 
(CPA) .   We  are  to  determine   u*(t)  =  U*(t,x,y)   such  that  the  following 
holds  : 


maximize  {G (T,x(T) ,y (T) )  +   L(t ,z ,u,V* (t ,x,y) )dt}  , 
U(t)  iQ 

subject  to:    x  =  f (t,x,y,V*(t ,x,y) ) ,  (CPA) 

rs*  f***  *n*  f*j  rs*    rsmf 

u(t)   is  unrestricted  scalar  control  variable, 

x  (t)  ^  a(t)   and  y  (t)  £  3(t)   for  all   t  €  [0,T], 
m  n 

with  scalar  terminal  condition   F(T,x(T) ,y (T) )  =  0. 

For  (CPA)  we  assume  that  the  first  time  derivative  of  y  (t) 

n 

• 

explicitly  contains  the  control  variable  u  and  that   (y  )   =  g   (t,x,y,u) 

n      n    ~  ~ 
u     u 

4-   0   along  an  optimal  trajectory.   Let  us  note  that  in  (CPA)  we  have 

(x  )   =0.   Thus,  variations  in  the  control  u  have  no  direct  influence 
m 
u 

on  a  trajectory  for  which  x  (t)  =  a(t).   Let  us  recall  that  we  have 

m 

assumed  that  in  (DG)  the  first  time  derivative  of   x  (t)   explicitly  con- 

m 

tains  the  strategic  variable  v   and  that   (x  )  (t,x,y,v)  ^  0   along  an 

m        ~  r~ 

v 
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optimal  trajectory.   Thus,  in  (CPA)  we  will  have  (since  in  (CPA)   x  (t)  ^ 

m 

+ 
ct(t)   is  a  "rule  of  the  game")   x   =  f  (t,x,y,V*(t  ,x,y))  ^  a(t)   when 

m    m   **<  ^*       »v  *w 

x  (t)  =  a(t)   where   f  (t  )   denotes  a  right  hand  limit. 

m  m 

There  are  two  cases  to  be  considered  in  developing  necessary  condi- 
tions of  optimality  for  (CPA) : 

(1)  x  (t)  >  a(t)   almost  everywhere  in  time, 

m 

X  X  XX 

(2)  x  (t)  =  a(t)   for  t  m    <:  t  <:  t  m.   with  t  m    <  t  m.  . 

m  entry        exit         entry    exit 

CASE  (1):   x  (t)  >  a(t)   almost  everywhere  in  time. 
m ■ 

We  will  apply  the  results  of  Speyer  [12]  (see  also  [17]  and  [24]) 
to  (CPA).   To  this  end,  we  define 

HA(t,z,pA,qA,u)  =  L(t,z,u,V*(t,z))  +  (pVf  (t  ,z  ,V*  (t ,  z)  ) 

+  (qVg(t,z,u)  -  n,(t){y  -3(t)},       (26) 

***   **»   •**       Z  n 


where 


n2(t) 


=  0   for  y  <  g(t), 
n 


;>  0  for  yn  =  6(t), 


and 

$A(T,z(T),oA)  =  G(T,x(T),y(T))  +  A (T,x(T) ,y (T) ) .         (27) 

According  to  the  results  of  Speyer  [12],  first  order  necessary  conditions 
of  optimality  for  (CPA)  are  as  follows: 


3 


H*   -      3L  ,  .   AXT  3& 


=  0   or  ^  +  (q  )   —  =  0,  (28) 


3u  3u    ~    3u 


<pV   .    .  |L   .    (   A   T   n.  _        AT  jfc  _    {|L  +        AT    " 

~  3x  ~  3x  ~  3x         l3v  **  3vJ       x 
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/•AvT     9L    ,   A.T  91    .  A.T  3&   ;3L    ,  A.T  31«  nNT     ,„rt. 

(q  )   -  -  9 (P  ) (q  )   T {—  +  (p  )   — }  V*  +  n  (A  )  ,    (30) 

A  dy    ~    dy    A  dy     dv    *-    dvJ   y     2  ~ 

a*A     a     t  .  ,  =  0   for   x  (T)  >  a(T), 

(p  (T))   =       -  v  (A  )   where  v   <  (31) 

3£(T)     X  ~  1  W  0   for   x  (T)  =  a(T), 

m 

^    (T))   =  z„fr\    ~  V?(A  >   where  v  <  (32) 

^(T)     2  ~  2Uo   for  yn(T)  =  3(T), 


HA(T)  +  |i--+  v^  a(T)  +  vA  6(T)  =  0,  (33) 

where  H  (T)   is  an  abbreviation  for  H(t=T,z(T) ,p  (T) ,q  (T),u*(T)).   We 

r**  *+m0  e+s 

will  make  copious  use  of  such  notational  conveniences.   We  also  have  the 
maximum  principle 

HA(u,V*(t,z))  *  HA(u*,V*(t,z)).  (34) 

A  A  A 

H  ,p  (t) ,   and  q  (t)   are  continuous  except  at  manifolds  of  discontinuity 

of  V*,   where  for  all  differentials   dt,   dx,   dy   along  the  manifold 

(HA+-HA")dt  -  (pA+-pA")Tdx  -  (qA+-qA")Tdy  =  0.  (35) 


When  y  (t)  <  3(t)   almost  everywhere  in  time,  the  above  becomes 
(except  at  manifolds  of  discontinuity) 

£+<sV2=0,  (36) 


As  noted  for  equation  (15) ,  these  results  hold  only  when  the  boundary  of 
the  state  space  (at  least  the  part  on  which  yn  =  3(t))  is  non-absorbing 
(see  [24]).   The  reader  is  referred  to  the  previous  footnote  for  the  state- 
ment of  terminal  multiplier  properties  for  an  absorbing  state  boundary. 
Henceforth,  in  order  to  contain  the  length  of  this  appendix  we  will  leave 
it  to  the  reader  to  supply  such  necessary  qualifications  for  conditions 
on  terminal  multipliers. 


A- 15 


,-A.T  3L         ,   A.T   31        ,   A.T    3&        r  3L         ,   A.T    31»  .... 

(P    )       -   -   a (P    )       a (^    )       a IT"  +    (P    )      T~ JV    »  ^37^ 

~  3x  ~  3x  ~  3x         l3v  *-  3vJ    x 

«w  <s»  **<  «~ 

•A.T  3L         ,   A.T   31        ,   A.T    3&        r3L         ,   A.T   3!i„.  ,,_,. 

(q    )       -   ~  T (p    )       a (^    )       a IT-  +    (P    )      T~JV    *  <38) 

~  3y  *~  3y  «*  3y  l3v  **  3v      V 

Using    (36)  ,   we  may  write   the  adjoint   equations   as 

/'AvT  3L         ,   A.T    3£        ,   ANT    3&        r3L         ,   A.T    3&n    ... 

<J>  >    -  -  ai  -  <£ )    t;  -  (q  )    t;  -  (t:  +  (q  )    tj!  u* 


^w 


.•A.T  3L         ,   A.T    3£        ,   A.T    3&        r3L         ,   A.T    3&oTT. 

<S,  >    =  "  t;  -  (p  )    t;  -  (q  )    tt  -  It:  +  (q  )    t^u* 

3f 


-e+CEV^V*.         (40) 


3v    ~    3vJ 


L 


Y  Y  Y        Y 

When  y  (t)  =  B(t)   for   t  n    £  t  ^  t  n.   with   t  n    <  t  n.  , 
■'n  entry        exit         entry    exit 

we  may  reduce  the  dimension  of  the  state  space  and  also  the  adjoint  variable 
space  (see  [17]).   Let  us  adopt  the  following  notation 

PT  =  (P  ,P  , . . .  ,P  )  ,  QT  =  (Q  ,Q  Q    )  , 

~      1   Z      m  ~      1   Z      n-l 

£T  =  (gi^'-'-'Vi^  z  =  ^i»y2"-"yn-i)' 

-T     T  -T\    ,  . 

z   =  (x  ,y  )  =  (x  , . . . ,x  ,y  , . . . ,y    ), 

~  *~  1      ml      n-l 


^w 


V  =  V(t,z)  =  V(t,x,y,y  =  B(t)). 

In  other  words,  in  the  reduced  state  space  the  state  variables  are 

x.  ,  .  .  .  ,x  ,y. , . . . ,y   . ,   and  the  corresponding  adjoint  variables  are  denoted 
1      ml      n-l 

A        A 
as  P   and  Q  .   When  V*   is  continuous  at  entry  to  the  boundary  of  the 

state  space  with  y  =  3(t),   the  corner  conditions  at  such  junctures  to 

Yn 

this  reduced  state  space  are  (for  example,  at  entry  with   t  =  t     )  as 

r  J  e    entry 

follows  for  a  first  order  SVIC  [24] 


and 


A- 16 


PA^tP)  =  £A(tt}'  (41) 

qA(t")  =  qA(t^),  (42) 


HA(t  )  =  HA(t+).  (43) 

e        e 


Assuming  that  the  optimal  path  is  not  tangent  to  the  boundary  of  the  state 

_         • 

space  at  entry  to  the  constrained  subarc  (i.e.   g  (t  )  -  (3(t  )  ^  0)  ,   it 

nee 

follows  that 

L(t+)        L(tj   +    (QA(t^))T{g"(t+)    -  i(tj} 
qA(t")    =  ^— = S ^1_^ 21_e 2^L_  (44) 

feCtf    "    3(te) 
where  we  have  adopted   the  notation 

p  (t  )  =  p  (t  ), 

and  (45) 

_A,  +N    -A,  +N 
Q  (t  )  =  q  (t  ). 

When  V*   is  discontinuous  at  entry  to  the  boundary  of  the  state  space  with 

y  =  6(t),   it  follows  from  adapting  the  argument  of  Berkovitz  [1]  on 
n 

discontinuous  "right-hand  sides"  to  the  usual  development  of  the  corner 
conditions  (see  [24]) 

pA(t")  =  At*),  (46) 

~   e    *"   e 

qA(t")  =  qA(t+)  +  pAn,  (47) 

~       e    *v   e     ~ 

and 

At")  =  HA(t+)  +  pB(te).  (48) 

By  (46)  through  (48)  and  the  non-tangency  assumption,  it  follows  that 
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.        L(t+)  -  L(t  )  +  (PA(t+))T{f(t+)  -  f(t  )}  +  QA(tt))T{g(tt)  "  g(t  )} 
A,  -%      e       e     ~e    ~  e    ~  e     ~e    **     e.         ^  e    /,o\ 
(t  )  =  _ . (A9) 

g  (t  )  -  6(t  ) 
n  e       e 

where  again  we  have  used  (45) .   It  should  be  noted  that  (49)  reduces  to 
(44)  when  V*   is  continuous  at  entry.   We  may  also  re-write  (46)  and  (47) 
(or  (41)  and  (42))  as 


and 


£A<0  =LA(<), 


iA<tI>  -aN#- 


(50) 


The  adjoint  equations  in  the  reduced  state  space  may  be  obtained  from 
(29)  and  (30)  by  use  of  (28)  to  eliminate  qA(t).   From  (28),  we  have 

3g 


A      1  /3L    ,-A.T  &l 


3u 

since  we  have  assumed   (g  )   ^  0  along  an  optimal  trajectory  for  (CPA) . 

n  u 

-A  A 

Since  q  (t)   is  denoted  as   Q  (t)   in  the  reduced  state  space,  this 

becomes 


A      1   (3L    , _A.T  3&) 


(51) 


Using  (51)  and  denoting  the  adjoint  variables  in  the  reduced  state  space, 
(29)  may  be  written  as 


lA.T 


3L 


,AXT 


9f 


.AVT 


»g 


c*r  -  -  ~  -  (r-v  tE  -  csr  £  +  Tafr  {£  +  (qa)t  ^ 


3x 


3x 


(N 

3x 

f8*n) 

3u 

3uJ 


l3v  ~  y      3vJ      x 


(52) 
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Similarly  for  (30) 


(QA)T  -  -  *i 


as 


n 


3y 


A  T  3f 

(PA)T  — 

9y 


3iLJ 


9g 


=  -  (QV  ^  +  -£-  {f  +  (QA)T  ^ 
y        3y     n 


3u 


3f 


L3v    ~    3vJ   y 


(53) 


It  should  be  noted  that,  to  be  precise,  in  the  reduced  state  space  we  have 


3V 


r9V 


3x 


k  t,y 

x.   for   i=l , . 

1     i/k 


■3x 


k  t,y 


,m 


x . 

l 


for   i=l, 
i  *   k 


,m 


etc.   However,  to  avoid  excessive  notation  we  will  not  be  precise  on  this 
point  and  hope  that  the  identification  of  the  independent  variables  when 

partial  derivatives  are  to  be  taken  is  clear  from  the  context.   When 

Y  Y  Y        Y 

/  \    „  /  \  r  n  n     .  .     n        n        . 

y  (t)  =  3(t)   for   t      £  t  £  t      with   t      <  t   .  ,   we  have 
n                 entry        exxt         entry    exit 

Y  Y 

K   "  ^^\  =   3(t),U*(t,£))  -  hit)      for   t^try  £  t  *  te^.t, 


except  at  manifolds  of  discontinuity  of  U*(t,z)  (i.e.  except  at  the  end 
points  of  the  interval) .  Differentiating  the  above  identity  with  respect 
to   x,   we  obtain 

g   +  g   U*  =  0, 

m     n   x 
x     u  ~ 


which  yields 


3g. 


3x 
— *■       ~ 

r3g 


iu 


(54) 
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since 


3u 


f   0   along  an  optimal  trajectory.   Similarly  we  obtain 

3V 


*      -3y  > 
U-  =  -  — . 


3u 


(55) 


Using  (54)  and  (55)  ,  we  may  write  (52)  and  (53)  as 


(pV  - 


_3L 
3x 


(PA)T  *f  -   (QA)T  £  -   {f  +  (QA)T  ft  U* 

~     3x     ~     dx     ldu     ~     duJ   X 


3f 


3v 


3vJ 


(56) 


<QA)T  - 


it  -  (PA)T 
3y 


3f 


3y 


,ANT 


9g 


'  3-l   ,T* 


<qV-   {f  +  (qV     u- 

~  ldu     ~     duJ   " 


-  {f  +  (pA)T  ft  v- 

l3v    ~    3vJ   y 


(57) 


X  X  XX 

CASE  (2):   x  (t)  =  o(t)   for   t  m    <;  t  <:  t  m.   with  t  m    <  t  m.  . 
m entry exit entry exit 

We  consider  the  optimal  control  problem  (CPA)  in  a  reduced  state 

space  in  which  the  state  variables  are  x1,...,x   .,  ty,  ,...,y    (also 

1      m-1   1      n 

-T    -T  T 
denoted  as   z  =  (x  ,y  ))   and  the  corresponding  adjoint  variables  are 

A   A 
denoted  as   P  ,  Q  .   Then  the  system  dynamics  are  given  by 


x  =  f(t,x,x  =a(t),y,V  (t,z)), 

rs*  r<**  **s         ni  **■*  *■*"* 


y  =  g(t,x,x  =a(t)  ,y,u)  , 

****         *^    ****      m       **»' 

_* 
with  V  (t,z)  =  V*(t,x,x  =a(t) ,y)   being  determined  by   f  (t,x,x  =a(t), 
*»         *^  m      ~*  m   ~  m 

y,V  (t,z))  =  a(t).   We  also  will  denote  U(t,x,x  =a(t) ,y)   as  U  =  U(t,z). 
~      ~  m 


In  the  reduced  state  space  we  define  a  Hamiltonian  as  follows 
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HA(t,z,PA,QA,u)  =  L(t,x,x  =a(t),y,u,V  (t,Z)) 


****         ***->  *-N- 


+  (PA)Tf(t,x,x  =a(t),y,V*(t,z)) 

»v      r*s  r^>        Jfl  t+*  i*s 


+    (QA)Tg(t,x,x  =a(t),y,u)  -  n9(t) {y  -B(t) }  , 


(58) 


where 


n2(t)  j 


=  0   for  y  <  B(t) , 
n 


:>  0   for  y  =  3(t) 
n 


According  to  the  results  of  Speyer  [12],  first  order  necessary  conditions 
of  optimality  for  (CPA)  in  CASE  (2)   are  as  follows: 


9HA   n      3L    /nA,T  3&   _ 
•5 —  =0  or  t-  +  (Q  )   y-=0, 

du  du     ~     du 


(59) 


(pA)T  .  .  JL  .  (pA)T  n.  _       A)T  H.  _   {|L  +  (pA)T  "  -*  , 

~  -      #w       -      ~       -     ldV      ~      dVJ    X 

dX  dX  dx  ~ 


3f 


9g 


3f 


(<rr    .  ^  _  (p^i  ^r  _  (QV  -=  -  {^  +  (pV  3v+  n9(An)\   (61) 

~        dy    ~    dy    ~    dy    ldv    ~    3vJ   y    2  ~ 

*Nrf  A^  *»*/ 


Using  (59)  ,  we  may  write  (60)  and  (61)  for  y  <  £(t)   as 

n 


(PV  -  -  Sk  -    o^I  "  .  (QA,T  !§  .  {|L  +  (  A  T  »§j  - 


3x 


3x 


3x 


3u 


3uJ   x 


3L 


:  9^ 


L3v    ~    3vJ   x 


(62) 


•A  T 

(Q  )  = 


3L 
3y 


3f 


9g 


(PA)T^-  (oV  — 

~    3y    ~    3y 


3g 


«V  ^}  5; 


9u 


3uJ 


3f 


-  {&  +  <?V  ^1  v 


_* 


3v 


3vJ 


(63) 
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-A   A  A 

H  ,  P  (t) ,   and   Q  (t)   are  continuous  except  at  manifolds  of 


r*s  r^r 


discontinuity  of   V*   and  possibly  at  junctures  to  the  boundary  of  the 

state  space  with   x   =  a(t).   Thus,  we  also  must  consider  necessary  condi- 

m 

tions  for  a  juncture  between  an  unconstrained  subarc  and  a  constrained 
subarc  on  the  state  boundary.   The  results  of  Speyer  and  Bryson  [17]  tell 
us  that 

pA(t+)  =  £(t;> , 

and 

QA(t*)  =  qA(t"),  (64) 

»■*■'   e    «~   e 

X 

where  t   =  t       and   t   denotes  a  left  hand  limit.   It  also  may  be 
e    entry        e 

shown  that 

HA(t+)  =  HA(t")  -  pA(t")a(t  ).  (65) 

e        e     m  e    e 

This  may  be  written  as 

L(t    ,x,x  =a(t^),y,u*(t^),V*(t^,i))   +    (PA(t^)  )Tf  (t    ,x,x  =a(t   )  ,y^,V*(t^,£) ) 
e~m  e<^  e  e  ^  ~e~e~m  e     **■•  e  ^ 

+    (QA(t+))\(t    ,x,x   =a(t    ),y,uMt+))    =  Ut    ,x,x=a(t),I)u*(tj)V*(t"z)) 
~       e       *»     e  ~    m  e~  e  e  ~    m  e~  e  e  «~ 

+  (EA(«?),jL<t..Ip>s,-"(te).lPv*(t;.i>)  +  (At^Vvy.^'V-i'^^e" 

-  piujict.).  (66) 

or    (using    (64)) 

p    (t"){f    (t    ,x,x  =a(t   ),y,V    (t~   z))    -  a(t   )}   = 
me        m      e  ~     me*-  e  ~  e 
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L(te'~'Xm=a(te)'I'uMtt)'V*(tt'^))    "  L  (te'^Xm=a(te)  »£'u*  (te}  »V*  (te  »S  } 

+    (PA(t+))T{f(t    ,x,x  =a(t   ),y,V*(t*  i))    -  f(t    ,x,x  =a(t   ),y,V*(t"  i))} 
~       e  ~     e  ~     ni  e     ~  e<~  ~     e  ~     m  e     ~  e  *** 

+    (QA(t+))T{g(t    ,x,x  =a(t   ),Y,u*(t^))    -  g(t    ,x,x  =a(t   ),y,u*(t"))}    .      (67) 
~e  >^     e  ~     m  e     *~  e  ~     e  ~     m  e*~  e 


When   the  state  trajectory  does  not   enter   the  boundary  of   the  state   space 

_*   _  _ 

tangentially,  we  have  that   f  (t  ,x,x  =a(t  ) ,y,V  (t  ,z))  -  a(t  )  f  0, 

m  e  ~  m    e  *"     e  ~        e 

A  - 
so  that  (67")  may  be  solved  (uniquely)  for   p  (t  )  .   Then 

m  e 

,        L(t^)-L(t")  +  (PA(t^))T{f(t^)-f(t")}+(QA(t^))T{g(t^)-g(t")} 

A(t-)  =  e e ~ e ~  e  ~  e ~ e fc  e  ~  e   ^      g 

me  c    ,    ~\         '  /      \ 

me       e 

When  f  (t  )  -  a(t  )  4   0  and  u*(t  )  =  u*(t  )  (i.e.  no  manifold  of 
m  e       e  e        e 

discontinuity  of  U*  at  entry  to  reduced  state  space) ,  (68)  becomes 

,       L(t+)  -  L(t")  +  (PA(t+))T{f(t+)  -  f (t~)} 
PA(t_)  =  S % 21_^ =!_§ 21_JL_.         (69) 

m  6  f  (O  -  a(t  ) 

me       e 

_*   _  _ 

When   f  (t  ,x,x  =  a(t  ) ,y,V  (t  ,z))  -  a(t  )  =  0,   then  the  juncture 
me~m      e*^     e~        e 

A  - 
conditions  (64)  and  (65)  (i.e.  (67))  do  not  determine  p  (t  ). 

m  e 


To  summarize,  we  have  for  (CPA) 


for  x  (t)  >  a(t).  y  (t)  <  8(t)   almost  everywhere  in  time: 
m *-n s 

,-A,T     3L    ,  A.T  8~   ,  A.T  ?fe   ;9L    ,  A.T  *\      * 
(p)   -  ~  -Z (p)   t (q  )   "8 \t-  +    (q  )  -T-)   U 

*-  9x      *^     dX      ~      X     ldU      A  dUJ    X 


"  $  +  <£A>T  S  <  •  <™> 
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•A-T      3L    ,  A.T  9~    ,  A.T  3&    r3L    ,  A.T  3&,   * 

(q  )   "  -  a (P  )   T7  "  (q  )   a IT"  +  ^  )   T~J  u 

oy     *~     dy     •*-     3y    ldu     ~    du    y 


Y  Y  Y        Y 

for  y.,(t)  =  B(t)   for  t.n    <;  t  s;  t  n.   with  t  n    <  t  n. 
"Hx entry exit entry exit 

and  x  (t)  >  a(t)   almost  everywhere  in  time: 


-m- 

3x         x~  '       3x 


(pv  .|t.(pv^(QV^-^+(Qv^u-: 


{£*tf>TSK- 


(qV  .  .  JL  .    (pA  T   H.  _       AT  !&  .  {|L  +       AT  %    -t 

~  .-  *»*  ~-  ~  _-  ldu  ~  duJ       y 

dy  dy  3y  ~ 

#-^  *>^  fw 

-   it  +  (IA>T  gv   Vi    .  (73) 

X  X                              XX 
e-                /    \            /    \       r               ni                             m              .,m  m 
for     x    (t)    =  a(t)      for      t              £  t  £  t              with      t               <    t 
m entry exit entry exit 

and  y(t)  <    3(t)   almost  everywhere  in  time: 

(PV  .  .  it  .     pA  T  f£  .        AT  |  .  (|t  +    (QA)T  %    -* 

~  —  ~  —  ~  dX  dU  «~  dU  X 

dX  dX  ~  ~ 

-tf^LVljvi,  (74) 

(QA)T  =   -   9L   -    (PA)T  !~  -    (QA)T  ^  -  {9L  +    (QA}T  ^   0* 

V)l  ;  3y        K~  }      3y         ^  ;       3y  l3u         ^  '       3u;      y 

_  ||L  +  (pA)T  ^  ^*  (75) 

l3v  ~  '      3vJ      y 
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5.2.   Necessary  Conditions  for  Control  Problem  B. 

We  also  consider  a  minimum  problem  (denoted  as  Control  Problem  B, 
(CPB))  associated  with  (SPC) .   We  are  to  determine  v*(t)  =  V*(t,x,y)   such 
that  the  following  holds: 


minimize  {G(T,x(T) ,y (T) )  + 
v(t) 


L(t,z,U*(t,x,y) ,v)dt}, 


subject  to:       x  =  f(t,x,y,v),  (CPB) 

r**  *>*  r*~>    *■»*/ 

y  =  g(t,x,y,U*(t,x,y)), 

v(t)   is  unrestricted  scalar  control  variable, 

x  (t)  :>  a(t)   and  y  (t)  <:  B(t)   for  all   t  £  [0,T], 
m  n 

with  scalar  terminal  condition  F (t ,x, (T) ,y (T) )  =  0. 

For  (CPB)  we  assume  that  the  first  time  derivative  of   x  (t) 

m 

explicitly  contains  the  control  variable  v  and  that   (x  )   =  f   (t,x,y,v) 

m  v    m    ~  ~ 
v 

^  0   along  an  optimal  trajectory.   Let  us  note  that  in  (CPB)  we  have 

(y  )   =0.   Thus,  variations  in  the  control  v  have  no  direct  influence 
n  v 

on  a  trajectory  for  which  y  (t)  =  $(t).   Let  us  recall  that  we  have 

assumed  that  in  (DG)  the  first  time  derivative  of   y  (t)   explicitly  con- 

n 

tains  the  strategic  variable  u  and  that   (y  )   =  g   (t,x,y,u)  ^  0  along 

n  u    n    "w  ~ 
u 

an  optimal  trajectory.   Thus,  in  (CPB)  we  will  have  (since  in  (CPB)   y  (t) 

*  + 

£  B(t)   is  a  "rule  of  the  game")   y  =  g  (t,x,y,U*(t  ,x,y))  £  3(t)   when 

n    n   *+*  **•>  **•>  *** 

y  (t)  =  6(t)   where  g  (t  )   denotes  a  right  hand  limit. 

There  are  two  cases  to  be  considered  in  developing  necessary  condi- 
tions of  optimality  for  (CPB) : 
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(1)  y  (t)  <  3(t)   almost  everywhere  in  time, 

n  Y  Y  Y        Y 

(2)  y  (t)  =  B(t)   for  t  n    £  t  <;  t  n.   with  t  n    <  t  n.  . 

n  entry        exit         entry    exit 

CASE  (1):   Yn(t)  <  &(t)   almost  everywhere  in  time. 

As  before,  we  will  apply  the  results  of  Speyer  to  (CPB) .   To  this 
end,  we  define 

ODD  R   T 

H  (t,z,p  ,q  ,v)  =  L(t,z,U*(t,z),v)  +  (p  )  f(t,z,v) 


fs  r** 


+    (qB)Tg(t,z,U*(t,z))  -  n,(t){x  -a(t)},     (76) 
. —   ^~   —      ***  i     m 


where 

,  =  0   for  x  >  a(t)  , 


m 


nx(t) 


k  0   for  x  =  a(t) , 
m 


and 


$B(T,z(T),aB)  =  G(T,x(T),y(T))  +  aBF(T,x(T) ,y(T) ) .         (77) 


According  to  the  results  of  Speyer  [12],  first  order  necessary  conditions 
of  optimality  for  (CPB)  are  as  follows: 

37"  =  °    °r   37  +  {l}      "37  =  °'  (78) 

,-B.T     3L    ,   B.T  91   ,  B.T  3&  r3L  ,  .   B.T  3&n   *      ,.m.T 

(P  )   ■  ~  "5 (P  )   a (^  )   a IT"  +  (cl  >   T7J  Uv  +  ni  (A  >  »    (79) 

*-        3x    *-    3x    ~    3x  l3u    ~    3uJ   x    1  <~ 

e+*  ^  ***  *^ 

.•B.T      3L    ,  B.T  3~    ,  B.T  3^    r3L    ,  B.T  3&.   *       f      . 

(q  )     ■  -  •; —  (p  )     a —  (q   )  a —   IT"  +  ^  )  T~J  U„'     (8°) 

~        3y    *~    3y    A  3y    l3u    ~    3uJ   y 
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,=0      for      x    (T)    >   a(T), 

(£  (T))    =  alTcfy  "  vi(i}      where    vi  (81) 

'  ;>  0   for  x  (T)  =  a(T) , 

m 

B  /=0   for   yn(T)  <  a(T), 

(^B(T))T=^k-V2^)T  Whe-   ^2  (82) 

H   for   yn(T)  =  B(T), 

B       3$      B*        B» 
HD(T)  +  ~-  +   v*a(T)  +  v!Jb(T)  =  0.  (83) 

We  also  have  the  minimum  principle 

HB(U*(t,z),v)  £  HB(U*(t,z),v*).  (84) 

R     R  R 

H  ,   p  (t) ,   and  q  (t)   are  continuous  except  at  manifolds  of  discontinuity 
of  \J* ,      where  for  all  differentials   dt,   dx,   dy   along  the  manifold 

/nB+  ,TB— N  ,     /  B+   B— .T  ,     .  B+   B— .  T  ,     _.  /nr\ 

(H   -H   )dt  -  (p   -p   )   dx  -  (q   -q   )   dy  =  0.         (85) 

When  x  (t)  >  a(t)   almost  everywhere  in  time,  the  above  becomes 
m 

(except  at  manifolds  of  discontinuity) 

£  +  (£,B>T  1  *  °>  <86> 

(pV  -  -  f-   -    (PB)T  1  -  (qV  '£  -    {f  +  (,V  UJ1  0*       (87) 

~        3x    ~    3x    ~    3x    L3u    ~    3uJx 

0**  r*s  ***  *>*f 

,-B.T     9L    ,  B.T  31   ,  B.T  3&   r  9L    ,  B.T  3gi   *      /Qa* 

(q  )  -  -  -r (p  )  t (q  )  t hr  +  (q  )  t42-}  u  .    (88) 

~        9y    *-    9y    ~    9y    l9u    «*    9uJ   y 

Using  (86),  we  may  write  the  adjoint  equations  as 

.•B.T      3L    ,  B.T  31    ,    B.T  3^    r9L    ,  B.T  3^   * 

(p  )   =  -  1 (p  )  1 (q  )   T |t-  +  (q  )   — }  U 

*~        9x    ~    9x    ~    9x    l9u    *    9uJ   x 

/*»*  /-v/  ^w 

-  {t±+  (pV-Jt!  v*  .  <89> 

L3v    ~    3vJ   x 


r^ 
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,-B.T  9L  ,    B.T    91  ,    B.T    3&  ,9L         ,    B.T    3&o       * 

(q  )     -  -  -r-  -  (p  )     ■=-  -  (q  )     t-  -  (t-  +  (q  )     t-}  U 

~  dy  ~         dy  ~         dy  ldu  **»         duJ      y 

*>*  ^N/  ^W  ^ 


X  X  XX 

When     x    (t)    ■  a(t)      for      t  m  £  t  £  r  m.        with      t  m  <    t  m.    . 

m  entry  exit  entry  exit 


we 


may  reduce  the  dimension  of  the  state  space  and  also  the  adjoint  variable 
space  (see  [17]).   Let  us  adopt  the  following  notation 

pT  -  CrP2'""Vi)'  QT  -  (QrQ2,....Qn). 

-T  -T 

£  =  (f1,f2,...,fm_1),  x  =  (x1,x2,...,xm_1), 

U  =  U(t,z)  =  U(t,x,x  =a(t),y). 

~        *^  m      ~ 

In  other  words,  in  the  reduced  state  space  the  state  variables  are 

x_  , . . . ,x    ,   y  ,  . . . ,y  ,   and  the  corresponding  adjoint  variables  are  de- 

R     R 

noted  as  P  ,   Q  .   When  U*   is  continuous  at  entry  to  the  boundary  of 
the  state  space  with  x  =  a(t),   the  corner  conditions  at  such  junctures 

xm 

to  this  reduced  state  space  are  (for  example,  at  entry  with   t  =  t     ) 

r  e    entry 

as  follows  for  a  first  order  SVIC  [24] 

£.B<0  -EB0-  (91) 

aB<0-aB0«  <»« 


and 


HB(t   )  =  HB(t  +).  (93) 

e         e 


Assuming  that  the  optimal  path  is  not  tangent  to  the  boundary  of  the  state 

_        • 

space  at  entry  to  the  constrained  subarc  (i.e.   f  (t   )  -  a(t  )  ^  0) ,   it 

J  m   e        e 

follows  that 
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L(T  +)  -  L(t   )  +  PB(t+)){f(t  +)  -  f(t   )} 
d.-s      e        e     ~   e    ~  e     ~  e 

P  (t   )  =  ; 

me  r    /      ~\  /      \ 

f  (t   )  -  a(t  ) 
me        e 

where  we  have  adopted  the  notation 

„B,   +N    -B,  +, 
I   (te  }  =  I   (te  }> 

and  (95) 

_B,  +,     B.  +. 
a  (te  >  =  &(te  >' 

When  U*   is  discontinuous  at  entry  to  the  boundary  of  the  state  space 

with  x   =  a(t),   it  follows  from  adapting  the  argument  of  Berkovitz  [1] 

m 

on  discontinuous  "right-hand  sides"  to  the  usual  development  of  the  corner 
conditions  (see  [24]) 

Z^e^    =  £B(te+)  +  ^  (%) 

qB(t  ")  =  qB(to+),  (97) 

~   e     *«   e 


and 


HB(t   )  =  HB(t  +)  +  pa(t  ).  (98) 

e         e         e 


By    (96)    through    (98)    and    the  non-tangency   assumption,    it    follows    that 

_  L(t^)-L(t")+(PB(t^))T{f(t+)-f(t")}+(QB(t^))T{g(t^)-g(t")} 

B,   -N  e  e~e  ~     e     ~     e  ~       e  ~     e     ~     e        ,nn\ 

pm(te)    =  ' 77^ 77~, '(99) 

fm(te)    -  a(te) 

where  again  we  have  used  (95)  .   It  should  be  noted  that  (99)  reduces  to 
(94)  when  U*   is  continuous  at  entry.   We  may  also  re-write  (96)  and  (97) 
(or  (91)  and  (92))  as 

P  (te  )  -  P  (te  ), 

and  (100) 

B /   — N    „B .   +N 
q  (t   )  =  Q  (t   ). 
~   e     ~   e 


A-29 


The  adjoint  equations  in  the  reduced  state  space  may  be  obtained  from  (79) 

u 

and  (80)  by  use  of  (78)  to  eliminate  p  (t).   From  (78),  we  have 

m 


B       1   r3L    ,-B.T  3~i 
'm  =  "  3f~  ^   +  (E  }   3^ 


m 


3v 


since  we  have  assumed   (f  )  /  0   along  an  optimal  trajectory  for  (CPB) . 

— R  R 

Since  p  (t)   is  denoted  as  P  (t)   in  the  reduced  state  space,  this 

becomes 


1       r3L         ,„BNT 


3f 


e+<?va  • 


(101) 


m 


3v 


Using  (101)  and  denoting  the  adjoint  variables  in  the  reduced  state  space, 
(79)  may  be  written  as 


(pBjT  .  .  tt  .  (pB)T  fl  .  {QB,T  'J.  .  {|t  +  (QB)T  %   - 


3x 


3x 


3x 
3f 


3u 


m 


3x 


3uJ   x 


3f 


+  —  .   Jit  +  (pB  T   ~ 

r3f  ^  l3v    ~  ;   3vJ 
m 


3v 


(102) 


Similarly  for  (80) 


(qV  - 


»k   -  (PB)T  "1  .  ((JB  T  f|  _  {|L  +  (QBjT  %    „* 

3y    v~  '   3y    v£  }      3y    l3u    ~    3uJ   y 


f3f 


m 


+ 


Lay 


3f 
t 

3v 


BXT 


3f 


1£+<EV.3 


m 


(103) 


v.  ./ 
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It  should  be  noted  that,  to  be  precise,  in  the  reduced  state  space  we 
have 


f9U 


rd\J 


t,y 

x.  for  1=1, . . . ,m-l 
1  i^k 


t,y 

x.  for  i=l , 
1  i#c 


.  ,m 


etc.   However,  to  avoid  excessive  notation  we  will  not  be  precise  on  this 
point  and  hope  that  the  identification  of  the  independent  variables  when 
partial  derivatives  are  to  be  taken  is  clear  from  the  context.   When 


X 


X 


m 


m 


X 


X 


m 


x  (t)  =  a(t)   for   t      S^t  £  t  ***.    with   t      <  t  "' .  ,   we  have 
m  entry        exit         entry    exit 


X  X 

x  =  f  (t,x,x  =a(t) ,y,V  (t,z))  =  a(t)   for   t      £  t  £  t     , 

m    in   ~  m      ~      ~  entry        exit 


-*   - 

except  at  manifolds  of  discontinuity  of  V  (t,z)   (i.e.  except  at  the  end 

points  of  the  interval) .   Differentiating  the  above  identity  with  respect 
to   x,   we  obtain 


which  yields 


3f 


m 


f   +  f  V-  =  0, 
m-    m  x 
x     v  ~ 


3f 


m 


-*       _!~ 

VX  "      rdf 


(104) 


m 


3v 


since  - —  ^  0  along  an  optimal  trajectory.   Similarly  we  also  obtain 

o  V 


V-  = 

y 


r3f  >| 
m 

3y 

r3f  > 
m 

3v 

(105) 
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Using  (104)  and  (105),  we  may  write  (102)  and  (103)  as 


(pB)T  .  .  3L  _   B  T  "A  _    (  B  T  *  _   {|L  +   B  T  %    - 

~        »-    ~    „-    ~    _-    l3u    ~    duJ   x 

3x  dx  3x  *~ 

rv*  #N^  *^ 

(qV  .  .  |L  _   B  T  H.  _        B  T  !§  _  {|i  +  (QB,T  !&}  - 

~        3y    ~    3y    ~    3y    l3u    ~    3uJ   y 

3f 

-  lir  +  (pB)T  irl  v*    •  do?) 

L3v    ~    3vJ   y 

Y  Y  Y        Y 

CASE  (2):   y  (t)  =  B(t) for   t  n    £  t  £  t  n.   with  t  n    <  t  n.  . 
•Hx — entry exit entry exxt 

We  consider  the  optimal  control  problem  (CPB)  in  a  reduced  state 

space  in  which  the  state  variables  are  x1 , . . . ,x  ,yn,...,y   1   (also  denoted 

1      ml      n— 1 

-T     T  -T 
as   z  =  (x  ,y  ))   and  the  corresponding  adjoint  variables  are  denoted  as 

R    R 

P  ,  Q  .   Then  the  system  dynamics  are  given  by 


***^     *^ 


x  =  f(t,x,y,y  =6(t),v), 

rs/      r+s  r**    ***>         XX 

y  =  £(t,x,£,y  =3(t),U  (t,z)), 

_*     _       .        _ 

with     U    (t,z)    =  U*(t,x,y,y  =3(t))      being   determined  by     g    (t,x,y,y  =3(t), 
~  ~  ^     n  n~~n 

—A 

U  (t,z))  =  B(t).   We  also  will  denote  V(t,x,y,y  =B(t))   as  V  =  V(t,z). 

In  the  reduced  state  space  we  define  a  Hamiltonian  as  follows 
HB(t,z",PB,QB,v)  =  L(t,x,y,y  =S(t),U  (t,z),v) 

+  (PB)Tf(t,x,y,y  =3(t),v) 

r^f  ***  *-*    *"w    XX 

+  (QB)Tg(t,x,y,y  =3(t),U*(t,z))  -  n  (t){x  -a(t)  }  , 

*n^      *w      rs*    r>^         XX  **■'  -L  til 


(108) 
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where 


nx(t) 


0   for  x  >  a(t) , 

m 


^  0   for   x   =  a(t) . 

m 


According  to  the  results  of  Speyer  [12],  first  order  necessary  conditions 
of  optimality  for  (CPB)  in  CASE  (2)  are  as  follows: 

TT-.-°    or  &  +  <lY  H  *  "■  (109) 

(pB)T  .  _  |L  _  (pB  T  H.  _        B  I  ft  .  {|i  +  (  B  T  %    g*  +  -   .  I 

~         3x     «~     3x     ~     3x    l3u     ~     3uJ   X     1  ~ 

(qB,T  .  .  3L  _  (pB)T  f£  _    B  T  ft  _  {|L  +    B  T  ft  ?  j 

~         -    ~     -  -    ldu    ~    du   y 

3y         3y         3y  ~ 

Using  (109),  we  may  write  (110)  and  (111)  for   x   >  a(t)   as 


m 


(PB>T  .<_&.•_    (pB  T  n.  _        B  T  fft  _  ,BL  +   B  T  ft  fl* 

~  '       9x    ~  ;   3x    VX  '      3x  i3u    ~    3uJ   x 

/V  ^W  A^  ^W 

-  it +  <LB>T  §  \  ■         <"2> 

(qB,T  .  _  JL  -  (PB)T  ft  .  (QB)T  f&  .  {|L  ;   B  T  |£}  g* 

~    „-    ~    ~-  l3u    ~    3uJ   y 
3y.        3y         3y  ~ 

r*~*  /v  *>*> 


— R     R  R 

H  ,   P  (t) ,   and   Q  (t)   are  continuous  except  at  manifolds  of 
discontinuity  of   U*   and  possibly  at  junctures  to  the  boundary  of  the 
state  space  with  y  =  B(t).   Thus,  we  also  must  consider  necessary  condi- 
tions for  a  juncture  between  an  unconstrained  subarc  and  a  constrained 
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subarc  on  the  boundary.   The  results  of  Speyer  and  Bryson  [17]  tell  us 

that 

t^B  ,   +.     B  ,  —  v 

I   (te>  =  ?  (te}' 

and  (114) 

^B  .  +>    — B  ,  — . 
Q  (t  )  =  q  (t  ), 

~   e    ~   e 

Y 

where  T  =  t       and   t    denotes  a  left  hand  limit.   It  also  may  be 
e    entry        e 


shown  that 


HB(t+)  =  HB(t  )  -  qB(t  )3(t  ).  (115) 

e        e     n  e    e 


This  may  be  written  as 

L(t    ,x,£,y -3(t   ),U*(t+  z),v*(tt))   +    (PB(t+))Tf(t    ,x,£,y  =3(t   ),v*(t+)) 
e  o^  ~     n  e  e  ~  e  «■«       e       ^     e  ~  *~     n  e  e 


+    (QB(t^))Tg(to,x,y,y   =3(t    ),U*(t^,z))    =  L(t    ,x,y,y   =B(t   ),U*(t    ,z),v*(t    )) 
~       e       **»     e  **•  **    n  e  e  ***  e  ~  ~     n  e  e  *»"  e 


+    (pB(t    ))Tf(t    ,x,y,y   =0(t    ),v*(t    ))    +    (qB(t    ))Tg(t      x,y,y   =3(t    ),U*(t,z)) 
~       e       «~     e  ~  ^     n  e  e  ~       e       »>»     e  ~  ~     n  e  ~ 

-qB(t")6(t    ),  (116) 

n     e  e 

or    (using    (114)) 

qB(t"){g    (t    ,x,y,y  =3 (t   ) ,U* (t~   z) )    -   3(t   )}   = 
nenen  e  e  ~  e 


L(t     x,y,y  =3(t   ) ,U*(t+   z) ,v*(t+))    -  L(t    ,x ,y ,y  =3 (t   ) ,U*(t    ,z) ,v*(t   )) 
e  ~  ~     n  e  e  ^  e  e  ~  ~     n  e  e  ^  e 


+    (PB(0)T{£(te,x,£,yn=B(te),v*(^))    -£(te,x,£,yn=3(te),v*(t^))}  + 


(QB(tt))T{g(t    ,x,y,y   =3(t    ),U*(t+,z))-g(t    ,x,y,y   =  3(t    ),U*(t    ,z))}.  (117) 

r^e  ^^e*^^^n  e  e  *»*  e^^n  c  e  ^ 
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When  the  state  trajectory  does  not  enter  the  boundary  of  the  state  space 

— *   —  —     • 

tangentially,  we  have  that   g  (t  ,x,y,y  =B(t),U  (t  ,z))  -  3(t  )  ±   0,   so 

n  e  ^  *~  n  e  ~        e 

that  (117)  may  be  solved  (uniquely)  for   q  (t  ) .   Then  • 

n   e 

L(t^)-L(t")+(PB(t^))T{f(t^)-f(t")}+(QB(t+))T{i(t^)-i(t")} 
qB(t")  =  ^ 5 = § ^_§ S ~ e &  e  ~  e   ^    (ng) 

n   6  g  (t  )  -  B(t  ) 

n  e       e 

-    •  + 

When  g  (t  )  -  3(t  )  f  0   and  v*(t  )   (i.e.  no  manifold  of  discontinuity 
nee  e 

of   V*   at  entry  to  reduced  state  space) ,  (118)  becomes 

L(t+)  -  L(t")  +  (QB(t^))T{g(t^)  -  i(t")} 
B.-N       e       e     ~   e    ~  e    ~  e 
q  (t  )  = . 

gn(te-)  -  «y 

JL 

When  g  (t  ,x,y,y  =  3(t  ) ,U  (t  ,z))  -  3(t  )  =  0,   then  the  juncture  condi- 
n  e  «~  *-'  n    e      e  ~        e 

R   — 

tions  (114)  and  (115)  (I.e.  (117))  do  not  determine   q  (t  ) . 

n      e 


To    summarize,    we  have   for    (CPB) 


for     x    (t)    >   a(t),    y    (t)    <    3(t)      almost   everywhere   in  time: 

m * — n - " 

,-B  T  3L         ,    B.T    91        ,    B.T    9&        r8L         ,    B.T    9%      * 

(P    )      ■   -  "5 (P    )       a (q    )      TZ  =    XT'  +    (<1    )      T~J    U 

~  3x  ~  dx  ~  dx  ldu  ~  du         X 


-   if  +    <PV  S   Vx    •  <1M> 


,-3.T  3L         -    B.T    9&        ,    B.T    9&        r  9L    ,     ,    B.T    9£i    TT* 

(q    )       =    ~  7 (P    )       a (^    )       a IT"  +    (^    )      T~J    U 

~  dy  ~  dy  ~  dy         ldu  »*  dvJ       y^ 


(F  +    (PB>T  ill   V*    '  <121> 

1 9v         ~         3vJ      y 
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Y  Y  Y  Y 

n  n  .  . ,  n  n 


for      y    (t)    =    B(t)       for      t               «£   t   ^   t   ".        with      t   '                 t 
— ti entry = exit entry exit 

and     x    (t)    >   a(t)      almost   everywhere   in   time: 

nv   -■— ■■  ....    ..-  —  .    .     .,  - .*.,---    .——■—■-  -— -,—  - 


-m- 


;B,T  3L         /T.BNT    91        ^BVT 


<pV  -  -  j£  -  (pV  5=  -  (qV  ?  -  {f  +  (QB)T  ^}  C* 

~  3x  ~  3x  ~  3x  ldu  ~  3uJ       x 

^  **^  *-^ 

3f 

-  fir  +  (pB)T  -pi  v*  ,  d22) 

l3v  ~  dVJ        X 

(qB)T  .   .  3L   _    (pB)T   »£  .        B)T  !&  .  {|L  +        B)I  !&,    - 

~  -  ~  —  ~  —  du  ~  du        y 

3y  3y  3y  ~ 

&S  «W  M 

X                             X                             XX 
,.                /\/\^               m                             m             #1m  m 
for     x    (t)    =  a(t)      for      t              £.  t  £  t              with      t               <    t 
m entry exit entry exit 

and     y    (t)    <    B(t)      almost   everywhere   in  time: 
*-n 

(pB)T  .   .   3L   .        B  T   H.  _       B  T   *l  _   {|L  +        B  T  g   £ 

~  „-  ~  ~-  ***  »-  du  ~  dU  X 

dX  dx  dx  ~ 

*s*  ^  r^ 

3f 

"   i¥±+    (pB)T^l   V-    ,  (124) 

ldv  ~  3vJ      x 

(qB)T   .   _  Mi  .    (PB,T  !|  -    (QB)T  !§  .   ilk  +    (QB)T  ^}    C* 

vii  ;  3y  ~  '      3y         v*i  3y        l3u  ~  3uJ      y 
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5.3.   Equivalence  of  Multipliers  in  the  Two  Problems. 

By  (70)  through  (75)  and  (120)  through  (125),  we  see  that   pA, 

A  R     R 

q   and   p  ,   q   satisfy  the  same  differential  equations  (appropriately 

modified  on  the  boundary  of  the  state  space) .   In  order  to  complete  the 

proof  of  the  equivalence  of  these  multipliers  it  remains  to  show  that 

they  take  on  the  same  terminal  values  and  possess  the  same  continuity 

properties  (both  interior  to  and  on  the  boundary  of  the  state  space) . 

Thus,  it  remains  to  discuss 

(1)  terminal  values  of  the  adjoint  variables, 

(2)  continuity  of  the  adjoint  variables  across  manifolds  of  discon- 
tinuity of  strategies  at  interior  point  of  state  space, 

(3)  juncture  conditions  at  boundary  of  state  space. 

5.3.1.   Terminal  Values  of  the  Adjoint  Variables. 

For  (CPA),  there  are  several  distinct  cases  that  must  be  considered 


CASE  (Al)  :   x„,(T)  >  ct(T),  y  (T)  <  g(T). 

m e — ti 

From   (31) ,    (32)  ,   and    (33)    we  have 


(^m)I.J»       +aA     3F 


3x(T)        "      3x(T)    ' 


(,A(T))T  =  r^v+aA     9F 


9y(T)        "      3y(T)    • 
L+(pA(T))Tf+(qA(T»Tg+||+aAf  =0. 


We  may  write  the  above  as 
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-I 

~-m 


-I 


%. 


DF 


3x(T) 
,T 


3F" 


D£(T) 

3F 
3T 


"pA(T) 


qA(T) 


A 


3G' 


3x(T) 

T 
3G 

ay(T) 


L    3T 


t=T 


(126) 


where  I  denotes  an  m  x  m  identity  matrix.  It  should  be  recalled  that 
we  have  assumed  that  the  optimal  path  is  not  tangent  to  the  terminal  mani- 
fold. This  non-tangency  assumption  (see  [3]  or  [15])  yields  that  there  is 
a  unique  solution  to  equations  (126) . 

X 

CASE  (A2):   x  (T)  -  a(T)   with x  (t)  =  a(t)   for   t  m    £  t  £  T  with 
m — m entry 

X 

t  m    <  T   and y  (T)  <  g(T) . 
—entry "ti 

In  this  case  we  have 


(PA(T))T 


3G 


3x(T) 


+  o 


A  3F 


3x(T) 


A     T 

(Q  (T))1 


3G 


3y(T) 


-A   3F 
+  a 


3y(T)  ' 


L+  (PA(T))Tf+  (QA(T))Tg+|f+aA|f  =0. 


We  may  write  the  above  as 


-I   , 
«~-m-l 


?T 


-I 

~n 


g 


T 

3F   -| 

3x(T) 

T 
3F 

3y(T) 

_3F 
3T 


PA(T) 


QA(T) 


-A 
a 


T 
3G 


3x(T) 
~  T 


3G' 


•i(T) 


-  L  - 


3G 

3T  Jt=T 


(127) 
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Again,  the  non-tengency  assumption  yields  that  there  is  a  unique  solution 
to  equations  (127) . 

It  should  be  pointed  out  that  there  are  two  ways  in  which  we  may 
think  of  equations  (127)  as  arising: 

(a)  the  usual  transversality  conditions  in  a  reduced  state  space  in 

which  the  state  variables  are  x, x   , ,y, , . . . ,y  , 

1      m-1   1      n 

(b)  the  usual  transversality  conditions  in  the  original  state  space. 

Let  us  now  show  that  (a)  and  (b)  are  equivalent.   By  (31)  through  (33), 

we  have  in  the  original  state  space  in  which  the  state  variables  are 

x    , . . . ,x      y,. . . ,y        that 
I  m  n 

,   A/mNXT  9G  A      8F  A.   m.T  ,.,  „ON 

<£. (T))   "Jim  +  a  Sgj!^if«»  •  (128) 

AT  9G  A      9F 

(a  (T))   -  aTmr +  a  97oo'  (129) 

A. 
HA(T)   +  ff-  +  v^a(T)    =  0.  (130) 

We  may  write  the  last   equation  as 

a 
AT  AT  9<t>  A* 

L  +    (pA(T))if  +    (qA(T))Xg  +^r+  vAa(T)    =   0. 


From  (27)  and  (128)  we  have 


A     3$A     A,.. 

Vl  =^TlYT-pm(T)- 

m 


Thus 


L  +  (pA(T))Tf  +  (qA(T))Tg  +  #  +  l^fi-  -  p*(T)}i(T)  -  0. 

m 

It  is  convenient  to  write  the  above  as 

A      A 
L+  (£A(T))Tf  +  pA(T){f  -ci(T)}  =  (qA(T))Tg  +  ^+  **    a(T)  =0.    (131) 

m 
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X 

Let  us  recall  that   x  =  a(t)   for   t  m    <:  t  £  T.   Thus,  we  have 

m  entry 

dx  X 

m  —           — **        —      •            m 

-r—  =  f  (t,x,x  =a(t),y,V  (t,z))  -  a(t)   for   t  f   <  t  aC  T, 

at  m   ~  m      ~      ~                entry 

or 

f   -  a(T)  =  0.  (132) 

m 


Using  (132) ,  we  then  obtain  from  (131)  that 


L+  (£W!+  (a*(T))\+!£+^»i.£(I)  -o.       ("3) 

m 


Recalling  that  the  state  variables  in  the  reduced  state  space  are  x  ,  .  .  .  , 
Xm-l'yl'""yn   (°r  l>   I}'   We  haV£ 

(**L)  =    fl*l)  +  f-i*i-]  .  ^  (T) 

^3T  J-  ^3T  '  ^x  (TV   -  dt      ' 

x(T),y(T)     di   x(T),y(T)     dXmU;  T,x(T),y(T)   dt 

rw        *«w  *"w        As^  *w        Aw 

since  on  the  constrained  subarc  we  have  x  =  x  (t) .   Hence,  in  the  reduced 

m    m 

state  space  (133)  becomes 

L+  (pA(T))Tf  +  (q\T))V  (|^)  =0. 

~    ~  ^   x(T),£(T) 

or 

L  4-  (£A(T))Tf  +  (qA(T))Tg  +  (f£)  +  0A(ff)  =  0.    (134) 

*      ~    ~  3Tx(T),y(T)       9Tx(T),y(T) 

*«w        Aw  *w        Aw 

Making  the  identifications  PA(T)  =  pA(T) ,  QA(T)  =  qA(T) ,   and  aA  =  aA, 

*«w  Aw  /%*  /n* 

we  see  that  (127)  follows  from  (128),  (129),  and  (134). 
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CASE  (A3):   x  (T)  =  a(T)   but  x  (t)  >  a(t)   for   t  €  (T-6,T)   and 
_m m 

y  (T)  <  g,(T). 

■"TT 


In  this  case  we  essentially  have  an  additional  terminal  constraint 

x  (T)  =  ot(T)   (and  the  condition  that   x  (t)  >  a(t)   for   t  €  (T-6,T) 
m  m 

where   6  >  0) .   From  (31),  (32),  and  (33)  we  have 


(p4(T))T=^+aA^--vVV' 


3x(T) 


3x(T)     1 


,  A/m,NT    3G      A   3F 
(q  (T))   =  g  —  +  a 


3y(T)       3y(T)  * 


L  +  (PA(T))T£+  (qA(T))\+|f  +  aA||  +  vAa(T)  =  0 


We  may  write  the  above  as 

,T 


-I 

0 

3F       - 
3x(T) 

PA(T)  " 

0 

-I 

T 
3F 

3y(T) 

qA(T) 

= 

fT 

T 
g 

3F 
3T    -- 

A 
a 

3G      A.Am.T-i 

37(ry+ vi(-} 


3G' 


3ytt) 


9G    A- 
L  "  3T  "  Via(T)- 


(135) 


t=T 


The  non-tangency  assumption  implies  that  equations  (135)  have  a  unique 
solution  which  we  denote  as 


PA(T) 


RA(vA) , 


qA(T) 


SA(vA) 


A 


■  iM) 
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AAA 
Hence,  the  terminal  multipliers   p  (T) ,   q  (T) ,   a   depend  (continuously) 

on  a  single  parameter,   v1 .   The  value  of  this  parameter,  however,  cannot 

be  determined  from  the  transversality  conditions  but  must  be  determined 

by  considerations  "in  the  large"  (see  Section  7.3  of  [5]).   The  parameter's 

value  is  chosen  so  that  the  system  is  steered  to  the  given  terminal  state 

(see  [25]  for  corresponding  one-sided  (optimal  control)  problem).   Thus 

A 
vn   is  chosen  so  that  x  (T)  =  a(T)   but   x  (t)  >  a(t)   for   t  6  (T-6.T). 
1  mm 

OTHER  CASES.   These  are  treated  analogously  to  the  above. 

For  (CPB)  ,  there  are  again  several  distinct  cases  that  must  be 
considered.   It  should  be  recalled  (see  Section  4  above)  that  the  optimal 
path  is  not  tangent  to  the  terminal  manifold. 


CASE  (Bl):   x_(T)  >  q(T),   y..(T)  <  3(T) 

m *■ *-n 


Similar  to  case  for  (CPA) ,  we  find  that 


-I 

0 

3F          -n 

3x(T) 

V(T) 

0 

-I 

~n 

T 
3F1 

3y(T) 

qB(T) 

*** 

fT 

A/ 

T 
g 

3F 

3T       - 

B 

a 

3G' 


3x(T) 
,T 


3G 


3y(T) 

3G 


-  L  - 


3T 


Jt=T 


(136) 


Again,  the  non-tangency  assumption  yields  that  (116)  has  a  unique  solution. 
By  (126)  and  (136)  we  see  that  pA(T) ,   qA(T) ,  a        and   p  (T) , 

R         R 

q  (T) ,   a   satisfy  the  same  set  of  equations  which  have  a  unique  solution. 
Hence,  in  this  case  we  have  shown  that 

PA(T)  =  pB(T), 

qA(T)  =  qB(T), 


B 
a  . 


(137) 
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CASE  (B2)  :   x  (T)  =  a(T)   with  x  (t)  =  a(t)   for   t      £_t£T_  with 

__ ^ ^ '. entry 

t  m    <  T and  _y  (T)  <  B(T). 

—entry -"a   **— - 

Similar  to  case  for  (CPA) ,  we  find  that  in  the  reduced  state  space 


~m-l 


3F' 


3x(T) 


-I 


3F 


f1 


~n   3y(T) 

9F 

3T   - 


PB(T) 


QB(T) 


-B 

a 


3G' 


3x(T) 
,T 


3G' 


3y(T) 
3G 


-  L  - 


3T 


t=T 


(138) 


Again,  the  non-tangency  assumption  in  the  reduced  state  space  yields  that 
(138)  has  a  unique  solution. 

By  (127)  and  (138)  we  see  that   PA(T) ,   QA(T) ,   aA  and   PB(T), 

R        — R 

Q  (T)  ,   a    satisfy  the  same  set  of  equations  which  have  a  unique  solution 
Hence,  in  this  case  we  have  shown  that 

PA(T)  =  PB(T), 

QA(T)  =  QB(T), 


-A     -B 

a        =     a    . 


(139) 


CASE  (B3):   x  (T)  =  ct(T)   but   x  (t)  >  q(t)   for   t  £  (T-6  ,T)   and 


-TTT 


-m 


y(T)  <  3(T). 

Tl 

In  the  necessary  conditions  of  optimality  for  (CPB) ,  we  have  v  ^  0, 


Similar  to  case  for  (CPA) ,  we  find  that 


-I 

^m 

0 

3r       — 
3x(T) 

"pB(T)~ 

0 

-I 

~-n 

T 
3F 

3y(T) 

qB(T) 

= 

fT 

T 
g 

3F 

3T    -J 

B 

a 

i-    3G      B.eTi 


3G" 


-  L 


3y(.T) 

3G    B- 
3?  "  Vla(T) 


t=T 


(140) 
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Again,  the  non-tangency  assumption  yields  that  (140)  has  a  unique  solution 
which  we  denote  as 

PB(T)  =  RB(vB), 

q  (t)  =  S  (v  ), 


B      VB.  BN 
°        =      1    (v1) 


As  was  the  case  for  (CPA) ,  the  value  of  v  cannot  be  determined  by  the 
transversality  conditions  at  t  =  T  but  is  determined  by  considerations 
"in  the  large." 

In  this  third  case,  the  problem  essentially  behaves  as  though  it 
had  two  terminal  equality  constraints.   With  more  than  one  terminal  equality 
constraint,  our  results  apply  to  differential  games  for  which 

pA(T)  =  pB(T), 
qA(T)  =  qB(T), 

a   =  a  .  (141) 

This  type  of  qualification  was  apparently  first  noted  by  Schmitendorf  and 
Citron  [16].   We  have  not  been  able  to  prove  (141)  for  differential  games 
with   p  >  1   (where   p  =  the  number  of  terminal  equality  constraints)  (nor 
has  anybody  else  apparently).   However,  it  seems  plausible  that  (141)  holds. 

Let  us  elaborate  about  the  plausibility  of  (141) .   In  (CPA)  we  have 
V*(t,x,y)   given,  and  we  determine  u*(t)  =  U*(t,x,y);   while  in  (CPB)  we 

have  U*(t,x,y)   given,  and  we  determine  v*(t)  =  V*(t,x,y).   Thus,  in 

. —  ~  *<*  t** 

both  problems  we  have  the  same  optimal  strategies   U*(t,x,y)   and  ^*^t'^X)' 

T  T 
For  a  given  initial  point  ^^niYn^      we  can  fi-nd  the  same  optimal  trajectory 
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(x*(t) ,y*(t) )  for  0  ^  t  ^  T  in  both  problems.  Hence,  we  would  expect 
to  be  able  to  find  the  same  multipliers  (they  may  not  be  unique)  in  both 
problems,  since  we  have  the  same  adjoint  equations,  transversality  conditions 

(as  well  as  system  dynamics),  and  optimal  trajectories  in  both  problems. 

AAA        B       B       B 
p  (T) ,   q  (T) ,   a   and   p  (T) ,   q  (T) ,  a        satisfy  the  same  equations 

(135)  and  (140)  (provided  that  we  equate   \>A  with   v?) .   If   pB(T),   qB(T), 

1         1        ~       ~ 

a        is  not  unique,  we  will  agree  to  pick  it  so  that  (141)  holds. 

OTHER  CASES.   These  are  treated  analogously  to  the  above. 

Thus,  we  have  shown  that  (137)  holds  in  CASE  (1)  and  (139)  in  CASE 
(2) .   In  cases  like  CASE  (3) ,  our  results  apply  to  differential  games  in 
which  (141)  (or  the  equivalent)  holds. 

5.3.2.   Continuity  of  the  Adjoint  Variables  Across  Manifolds  of  Discontinuity 
of  Strategies  at  Interior  Point  of  State  Space. 
Let  M(t,x,y)  =  0  define  the  manifold  of  discontinuity  and   t, 
denote  the  time  at  which  this  manifold  is  reached  along  a  particular 
trajectory.   Assuming  that  the  optimal  path  is  not  tangent  to  any  manifold 
of  discontinuity  at  an  interior  point  of  the  state  space,  standard  argu- 
ments (see  [3]  or  [15])  in  which  the  manifold  of  discontinuity  M  plays 
the  role  of  the  terminal  manifold  then  yield 

A/  -v     B,  -v 

E  (td}  =  l  (td}> 

and  (142) 

A,  -v     B,-N 
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Again,  the  non-tangency  assumption  yields  that  (140)  has  a  unique  solution 
which  we  denote  as 

B,  .     B,  B, 
p  (T)  =  R  (v  )  , 

qB(t)  =  SB(v^), 

B      VB,  B. 
*       =     I    (vx). 

As  was  the  case  for  (CPA) ,  the  value  of  v  cannot  be  determined  by  the 
transversality  conditions  at  t  =  T  but  is  determined  by  considerations 
"in  the  large." 

In  this  third  case,  the  problem  essentially  behaves  as  though  it 
had  two  terminal  equality  constraints.   With  more  than  one  terminal  equality 
constraint,  our  results  apply  to  differential  games  for  which 

pA(T)  =  pB(T), 
qA(T)  =  qB(T), 

A  B  /1/1N 

a       =     a    .  (141) 

This  type  of  qualification  was  apparently  first  noted  by  Schmitendorf  and 
Citron  [16]  .   We  have  not  been  able  to  prove  (141)  for  differential  games 
with  p  >  1   (where  p  =  the  number  of  terminal  equality  constraints)  (nor 
has  anybody  else  apparently).   However,  it  seems  plausible  that  (141)  holds. 

Let  us  elaborate  about  the  plausibility  of  (141) .   In  (CPA)  we  have 
V*(t,x,y)   given,  and  we  determine  u*(t)  =  U*(t,x,y);   while  in  (CPB)  we 
have  U*(t,x,y)   given,  and  we  determine  v*(t)  =  V*(t,x,y).   Thus,  in 

both  problems  we  have  the  same  optimal  strategies  U*(t,x,y)   and  V*(t,x,y). 

T  T 
For  a  given  initial  point   (^jyn)   we  can  finc*  the  same  optimal  trajectory 
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(x* (t) ,y* (t) )  for  0  ^  t  £  T  in  both  problems.  Hence,  we  would  expect 
to  be  able  to  find  the  same  multipliers  (they  may  not  be  unique)  in  both 
problems,  since  we  have  the  same  adjoint  equations,  transversality  conditions 

(as  well  as  system  dynamics),  and  optimal  trajectories  in  both  problems. 

AAA  R         R         R 

p  (T) ,   q  (T) ,   a   and   p  (T) ,   q  (T) ,  a        satisfy  the  same  equations 

(135)  and  (140)  (provided  that  we  equate  \h     with   v?) .   If   pB(T),   qB(T), 

1         1        <~       «■«- 

o        is  not  unique,  we  will  agree  to  pick  it  so  that  (141)  holds. 

OTHER  CASES.   These  are  treated  analogously  to  the  above. 

Thus,  we  have  shown  that  (137)  holds  in  CASE  (1)  and  (139)  in  CASE 
(2) .   In  cases  like  CASE  (3) ,  our  results  apply  to  differential  games  in 
which  (141)  (or  the  equivalent)  holds. 

5.3.2.   Continuity  of  the  Adjoint  Variables  Across  Manifolds  of  Discontinuity 
of  Strategies  at  Interior  Point  of  State  Space. 
Let  M(t,x,y)  =  0   define  the  manifold  of  discontinuity  and   t 
denote  the  time  at  which  this  manifold  is  reached  along  a  particular 
trajectory.   Assuming  that  the  optimal  path  is  not  tangent  to  any  manifold 
of  discontinuity  at  an  interior  point  of  the  state  space,  standard  argu- 
ments (see  [3]  or  [15])  in  which  the  manifold  of  discontinuity  M  plays 
the  role  of  the  terminal  manifold  then  yield 

A/  -v     B,  -N 

E  (td}  ■£  (V> 

and  (142) 

A/  "x     B/-x 

s^V  =s,(d}- 
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If  M  is  a  manifold  of  discontinuity  for  only  U* ,   then  in  (CPA) 
the  usual  corner  conditions  at  an  interior  point  of  the  state  space  yield 

A.  -v    A.  +N 
Z   (td}  =  ^(td}' 

and  (143) 

q  (t  )  =  q  (t  ). 

«~   a    ~   a 

Similarly,  if   M   is  a  manifold  of  discontinuity  for  only  V*,   then  in 

(CPB)  the  usual  corner  conditions  at  an  interior  point  of  the  state  space 

yield 

B  ,  - ,     B  ,  +. 

z  (V  -  z  (V» 

and  (144) 

B  /  - \     B  /  +x 

If  M  is  a  manifold  of  discontinuity  of  both  U*  and  V* ,   then 

we  have,  for  example,  in  (CPA)  that  [1]  for  all  differentials  dt,,   dx, 

d    ~ 

dy  along  the  manifold 

{HA(t^)-HA(t~)}dtd  -  {pA(t^)-pA(td)}Tdx  -  {qA(td)-qA(t^)}TdI=  0.   (145) 

5.3.3.   Juncture  Conditions  at  Boundary  of  State  Space. 

Let  us  consider  the  juncture  conditions  which  must  hold  between  the 

adjoint  variables  at  the  entrance  to  a  constrained  subarc  on  which  x  (t)  = 

X      X  X        X 

f*(t)   for   t  6  [t     ,t   .    with  t      <  t   .  .   Other  cases  are 
entry  exit  entry    exit 

x 

similarly  treated.   We  denote   t       as   t  . 

entry  e 

For    (CPA)   we  have  shown   that 

£A(t")    =  PA(t^),  (146) 

>»<       e         ~       e 

qA(tj    =   QA(0,  (147) 
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and 


HA(t  )  -  pA(t  )a(t)    =   HA(tV  (148) 

e     m  e    e        e 


Assuming  that  the  optimal  trajectory  does  not  enter  the  boundary  of  the 

state  space  tangentially ,  (146)  ,  (147)  ,  and  (148)  yield  a  unique  value 

for     p    (t~). 
m      e 

,  L(t+)    -   L(t")    +    (PA(t^))T{f(t+)    -   f(t~)}    +    (QA(t^))T{g(t^)    -   g(t")} 

A,    -x               e                 e            ~e«~e~e              ~       e          ~     e          ~     e 
p    (t    )    = _ 

me  r       /      ~\  /  \ 

f    (t    )    -  a(t    ) 
me  e 

When  there  is  no  manifold  of  discontinuity  of  U*   at  entry  to  the  reduced 
state  space,  the  above  further  simplifies  to 

,       L(t+)  -  L(t")  +  (PA(t+))T{f(t^)  -  f(t")} 
pA(t_)  =  £ S 2^-J! 21_£ 21_fL_.         (150) 

m    e  fm<0  -  »(t ) 

me       e 

For    (CPB)    we  have   shown   that    (for   a   first   order   SVIC)    assuming    that 

the  optimal   trajectory   is   not   tangent   to   the  boundary  of    the  state  space 

at   a  juncture  point 

pB(t")    =  PB(tJ>,  (151) 

~       e  <~       e 

R  L(t+)    -  L-(t")   +    (PB(t+))T{f(t~Vf(t+)}   +    (QB(t^))T{g(0-g(t")} 

o,-\  e  e  ~       e  ~     e     ~     e  ~e  X/     e     ~     e  /n 

P   (t   )   =  ,  (i: 

me  e 

and 

qB(t")    =  QB(t+).  (153) 

^       e  *+■>       e 

When  there  is  no  manifold  of  discontinuity  of   U*   at  entry  to  the  reduced 
state  space,  equation  (152)  further  simplifies  to 
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-        L(t*)  -  L(t")  +  (PB(t+))T  f(t+)  -  f(t~) 
p»(t;>  .  _<= ~   e    ~  e ~_^_ 

W  -  .(t.) 

— B   + 
In  the  original  state  space  we  have  (recall  that  we  denote  p  (t  )   as 

R   + 
P  (t  ) ,   etc.,  in  the  reduced  state  space)  when  U*   is  continuous  at 

entry 

B,  -N     B.  +, 

I  ^   =  I  Cte), 

and  (155) 

B,  -x    B.  +s 
~   e    ~   e 

The  above  yield  that  when  £  (t)  =  p  (t)   and   q  (t)  =  q  (t)   for 
t  <  t  £  T,   it  follows  that  provided  the  optimal  trajectory  is  not  tangent 
to  the  boundary  of  the  state  space  at  a  juncture  point 

A,  -v     B,  -N 
£  (V  =  £  (te}' 

and  (156) 

qA(t")  =  qB(t-). 
*~   e    ~       e 


A  B 

5.3.4.   Equivalence  of  the  Adjoint  Variables  X  (t)   and  X  (t) . 

* W ^ jg S 

It  is  convenient  to  denote 

(AA(t))T  =  ((pA(t))T,(qA(t))T),  (157) 

*-*~<  *^        *~~ 

R  A  B 

and  similarly  for   X  (t) .   The  equivalence  of   X  (t)   and   X  (t)   follows 

from  the  facts  that 

A       B 

(1)  on  the  terminal  manifold  we  have  X  (T)  =  X  (T) , 

(2)  as  we  work  backwards  from  the  terminal  manifold  to  the  first 
manifold  of  discontinuity,  both   XA(t)   and   XB(t)   satisfy  the 
same  differential  equation  so  that'  xA(t)  =  X^(t)      for   t  <  t  £  T 
where   t..   denotes  the  time  closest  to   T   such  that  either  U* 
or  V*   is  discontinuous, 
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(3)  working  backwards  across  the  first  manifold  of  discontinuity, 
we  have  X  (t..)  =  X  (t,), 

~     1      <N»     i 

(4)  in  this  manner  one  works  backwards  across  each  manifold  of  dis- 
continuity and  shows  that  X  (t)  =  X  (t)   for  0  <;  t  £  T. 

Thus,  (70)  through  (75),  (120)  through  (125),  (37),  (139),  (141),  (142), 

and  (156)  yield  that 

for  0  £  t  <i   T:        pA(t)  =  pB(t)  , 

and  (158) 

q  (t)  =  q  (t). 


6 .   A  Problem  with  More  General  Dynamics. 

The  above  development  also  applies  to  the  following  problem  (denoted 
as  "Problem  II")   with  slightly  more  general  system  dynamics. 


Problem  II. 

T 


maximize  minimize  {G(T,w(T) ,x(T) ,y (T) )  + 

U        V  ~  I 


L(t,w,x,y,u,v)dt} , 


0 


subject  to:  w  -   f (t ,w,x,y ,v) , 

x  -  g(t,w,x,y,u,v) , 
y  -  h(t,w,x,y,u) , 

u(t) ,  v(t)   are  unrestricted  (scalar)  strategic  variables, 

w(t)  ^  a(t)   for  all   t  6  [0,T]  (scalar  state  variable 

inequality  constraints) , 
y(t)  <;  B(t)   for  all   t  6  [0,T] 

with  scalar  terminal  condition  F (T ,w(T) ,x(T) ,y (T) )  =  0, 


* 


Considering  the  results  of  Sections  5.3.2  and  5.3.3,  we  see  that  the  subse- 
quent statement  applies  whether  the  manifold  of  discontinuity  lies  interior 
to  the  state  space  or  on  the  boundary. 
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where  we  assume  that  all  functions  are  smooth  enough  to  insure  the  existence 
of  all  partial  derivatives  required.   In  Problem  II  above,   x   is  an   n- 
vector  of  state  variables. 

It  will  be  convenient  to  denote  the  vector  of  state  variables  as 

T       T 
z   =  (w,x  ,y)  -  (w,x  ,x  ,...,x  ,y) . 
~       ~  l   z      n 

We  will  then  sometimes  write  terms  in  a  more  convenient  form  as 

L(t,z,u,v)  =  L(t,w,x,y,usv) . 

Before  we  state  necessary  conditions  of  optimality  for  Problem  II, 
let  us  state  the  assumptions  that  we  make  for  Problem  II: 

(1)  the  differential  game  has  value  and  a  saddle  point  in  pure 
strategies , 

(2)  the  solution  to  Problem  II  is  normal  (see  [  1  ]) , 

(3)  the  optimal  path  is  not  tangent  to  the  terminal  manifold  or  to 
any  manifold  of  discontinuity  of  U*  or  V*  or  to  the  boundary 
of  the  state  space  at  entry  to  a  constrained  subarc  which  lies 
on  the  boundary, 

(4)  to  insure  the  existence  of  value  for  the  differential  game  we 
assume** 


(a)  L(t,z,y,v)  =  L  (t,z,u)  +  L  (t,z,v), 

(b)  g(t,z,u,v)  =  g  (t,z,u)  +  g  (t,z,v), 


(5)   the  first  time  derivative  of  y(t)   explicitly  contains  the  strategic 

variable  u  and  that   (y)   =  h  (t,w,x,y,u)  ^  0  along  an  optimal 

trajectory;  we  further  assume  that   (y)   =  0,   i.e.   h  does  not 
depend  on  v  at  all, 


* 

The  development  of  these  necessary  conditions  is  similar  to  that  for 

Problem  I. 

** 

As  noted  above,  several  other  technical  conditions  must  be  satisfied  to 

guarantee  the  existence  of  value  and  of  a  saddle  point  in  pure  strategies 

to  Problem  II  (see  Chapter  6  of  [9]). 
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(6)   the  first  time  derivative  of  w(t)   explicitly  contains  the 

strategic  variable  v  and  that   (w)   =  f  (t,w,x,y,v)  ^  0  along 
an  optimal  trajectory;  we  further  assume  that  Tw)   =  0,   i.e. 
f  does  not  depend  on  u  at  all. 

Now  we  define 


T 
H(t,z,P,n,u,v)  =  L(t,z,u,v)  +  Xf (t,z,v)  +  p  g(t,z,u,v) 


+  qh(t,z,u)  -  ni(t){w-a(t)}  -  n9(t){y-(3(t)},      (159) 


where 


,  =   0   for  w  <  a(t)  , 
k  £  0   for  w  =  a(t)  , 


and 


n2(t) 


=  0   for  y  <  3(t), 
;>  0   for  y  =  6(t), 


and 


*(T,z(T),ff)  =  G(T,w(T),x(T),y(T))  +  aF(T,w(T)  ,x(T)  ,y  (T)  )  .      (160) 


In  order  that  the  strategy  pair   (U*,V*)   be  a  saddle  point  of  the  criterion 
functional  it  is  necessary  that  there  exist  unique  functions   X(t),   p(t), 
q(t)  ,  r\    (t)  ,   and  n?(t)   and  constants  a,   v  ,   and   v   such  that  the 
following  conditions  hold 

3H    _       3L  .   T  ~     oh   n  /i  ti  \ 

t—  =0  or   —  +  p  —  +  q  —  =  0,  (161) 

dU  3u    ~   3u       du 

3H   A       3L     3f    T  ~   _  /Tcn\ 

t—  =0   or   —  +  X  —  +  p  —  =  0,  (162) 

3v  3v     3v   ~  3v 

-X=H  =T~+X^—  +P  ^~+cl^ H,  ,  (163) 

w   3w     3w   ~  3w     3w    1 


In  other  words,  we  assume  that  w(t)  ^  a(t)   and  y(t)  ^  3(t)   are  the 
appropriate  types  of  first  order  SVIC's. 
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•T        „           3L        ,  3f          T   3~  3h 

-p      -   H  =  —  +  X—  +   p1  —  +  q  ~,                                       (164) 

~            x         3x           dx                ox  3x 

„           3L        ,3f          T   3&  ,  3h  /n^CN 

-q      =   H  =  —  +   X—  +p      —  +    q n_,  (165) 

y        3y          3y        ~     3y  3y          2 


^*,^     ■  vn      where     v      j 


=   0      for      w(T)    >   a(T) , 


3w(T)  1      "*""■  =      vi    )  * 

VW  '^0      for     w(T)    =a(T),  (166) 

a$  /  =  0      for     y(T)    <    3(T), 

q(T)  =  sTTtT  "  v2  where  V2  ) 

YU;  '^0   for   y(T)  -  B(T),        (168) 

H(T)  +  H  -  vxo(T)  -  v23(T)  =  0.  (169) 

Analogous  to  the  Weierstrass  condition,  we  also  have  the  max-min  principle 

H(u,v*)  £  H(u*,v*)  <;H(u*,v),  (170) 

which  holds  for  all  admissible   u  and  v.   H,  X(t),   p(t)  ,   and   q(t) 
are  continuous  functions  except  at  manifolds  of  discontinuity  of  both  U* 
and  V*,   where  for  all  differentials  dt,   dw,   dx,   dy  along  the  manifold 

(H+-H~)dt  -  (X+-X~)dw  -  (£+-£~)Tdx  -  (q+-q")dy  =  0.         (171) 

In  other  words,  at  both  entrances  to  and  exits  from  constrained 
subarcs  on  the  boundary  of  the  state  space,  we  have 

X(0    =    *<£>.    £(0  =  £<t+),    q(t")  =  q(t+),  (172) 


and 


H(t  )  =  H(t+),  (173) 

e       e 


* 
It  should  be  recalled  that  this  statement  must  be  modified  for  an  absorbing 

state  boundary. 
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except  possibly  at  manifolds  of  discontinuity  of  both  U*   and   V*, 
where  all  the  dual  variables  are  continuous  except  those  corresponding 
to  w  or  y  which  describes  the  boundary. 

The  above  first  order  necessary  conditions  of  optimality  for 
Problem  II  are  analogous  to  the  optimal  control  theory  results  of  Jacobson, 
Lele,  and  Speyer  [12] .   These  results  may  also  be  written  in  a  form 
analogous  to  that  of  optimal  control  theory  results  of  Berkovitz  [2]  and 
Gamkrelidze  (see  Chapter  VI  in  [14]).   This  is  done  by  introducing  new 
multipliers   A(t)   such  that 


X(t)  =  A1(t)  -  u1(t),  (174) 

p(t)  =  A_(t),  (175) 

q(t)  =  A3(t)  -  u2(t).  (176) 
It  is  well  known  that  we  may  then  make  the  following  identifications 


ni(t)  =  -  ut(t)   for   i  =  1,2.  (177) 


7 .   A  More  General  Type  of  State  Variable  Inequality  Constraint. 

It  is  of  considerable  interest  and  import  to  consider  a  more  general 
type  of  SVIC.   We,  therefore,  consider  the  following  problem  (denoted  as 
"Problem  III") . 
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Problem  III. 

T 

L(t,x,£,u,v)dt} , 
0 
subject  to:        x  =  f(t,x,y,v), 


maximize  minimize  {G(T ,x(T) ,y (T) )  + 

U        V 


y  -  g(t,x,y,u) , 

/W         0S*  tfS^   *N* 

u(t) ,  v(t)   are  unrestricted  (scalar)  strategic  variables, 


C.(t,x)  ;>  0   for  all   t  6  [0,T]      (scalar  state  variable 

1   »* 


C2(t,y)  ^  0   for  all   t  €  [0,T] 


inequality  constraints) , 


with  scalar  terminal  condition  F(T,x(T)  ,y^(T) )  =  0, 

where  we  assume  that  all  functions  are  smooth  enough  to  insure  the  existence 
of  all  partial  derivatives  required.   In  Problem  III  above,   x   is  an  m- 
vector  of  state  variables,  and  y   is  an  n-vector  of  state  variables. 
It  will  again  be  convenient  to  adopt  the  notational  conventions  (6)  and 
(7). 

We  again  make  the  assumptions  (1)  through  (3)  given  in  Section  6 
above  and  modify  (4)  through  (6)  as  follows: 

(4)  to  insure  the  existence  of  value  for  the  differential  game  we 

assume  L(t,z,u,v)  =  L  (t,z,u)  +  L  (t,z,v), 

~         1   <~       z   ~ 

(5)  the  first  time  derivative  of   C  (t,y(t))   explicitly  contains  the 


strategic  variable  u  and  that   (C„)  (t,x,y,u(t))  ^  0  along  an 


optimal  trajectory;  we  further  assume  that   (C„)   =  — 
r         J     J  2  v   3v 

E  0,   i.e.  C        does  not  depend  on  v  at  all. 

(6)   the  first  time  derivative  of  C  (t,x(t))   explicitly  contains 

1   *** 

the  strategic  variable  v  and  that   (C  )  (t,x,y,v(t))  ^  0  along 


/■*-•  r^> 


an  optimal  trajectory;  we  further  assume  that   (C  ) 

-ac,   3C,  ^  l   u 

1         _  • 

■  tt~     =0,   i.e.  C.   does  not  depend  on   u  at  all 


8u 


3t     8x  ~ 
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Unfortunately,  it  is  not  convenient  to  develop  necessary  conditions 

for  Problem  III  in  the  same  manner  in  which  they  were  developed  for 

Problem  I.   We  will  now  state  necessary  conditions  of  optimality  for 

Problem  III.   Although  a  proof  has  been  developed  for  these  conditions, 

it  will  be  given  elsewhere  in  the  future,  since  it  is  rather  lengthy. 

First  we  define 

T 
H(t,z,p,q,n ,u,v)  =  L(t,z,u,v)  +  p  f(t,x,y,v) 


<-w  r^>     y-w  r*~> 


+   qTg(t,x,y,u)  -  n  (t)C  (t,x)  -  n  (t)C  (t,y),     (178) 


where  for   i  =  1,2 


n±(t) 


=  0   for   (-1)  C.  £  0, 


;>  0  for  C.  =  0, 
x 


and 


*(T,z(T),a)  =  G(T,x(T),y(T))  +  aF(T,x(T) ,y(T) ) .  (179) 

In  order  that  the  strategy  pair   (U*,V*)   be  a  saddle  point  of  the 
criterion  functional  it  is  necessary  that  there  exist  unique  functions 
p(t),   q(t),   rii  (t)  ,   and  ru(t)   and  constants  v,  ,  v_ ,   and  a   such 

*»»        »v         ±  Z  1     z 

that  the  following  conditions  hold 

|^-0  or  |L  +  qT-~=  0,  (180) 

dU  dU    ~   dU 

9f 

!^  =  °  or  J^+Z   iV=  °>  a81) 

•T    «     9L+   T  914   T  ^      3C1  h„. 

'I    =\=J^+Z  JZ  +  1    te  -  ni  IT »  (182) 

*T    „     3L+  T  914   T  ^      9C2  ,___. 

-a  =  Hv  =  JZ  +  p  ^7  +  a  S7  -  VaT-         (183) 
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xr  /  =  0   for  C  (T,x(T))  >  0, 

z  <T>  ■  ^en  -  vi  mt)  where  vi  {  ,   (184) 

^  ;>  0   for   C. (T,x(T))  =  0, 
1   ~ 

sc  (  =  °  for  C2(T»L(T))  <  °» 

q  (T)  =  ,„?t^  _  vo  ,,,/t^   where   v0  <  (185) 


3y(T)    v2  3y(T)  v2 


:>  0   for   C.(T,y(T))  =  0, 


3C        3C 

H(T)  +H+  vi  ir+  v2  *r  =  0-  <186> 

Analogous  to  the  Weierstrass  condition,  we  also  have  the  max-min  principle 
(18).   H,   p(t),   and   q(t)   are  continuous  functions  except  at  manifolds 
of  discontinuity  of  both  U*  and  V*,   where  for  all  differentials  dt, 
dx,   dy   along  the  manifold  (19)  holds. 

The  above  first  order  necessary  conditions  of  optimality  for  Problem 
III  are  analogous  to  the  optimal  control  theory  results  of  Jacobson,  Lele, 
and  Speyer .   These  results  may  also  be  written  in  a  form  analogous  to  that 
of  optimal  control  theory  results  of  Berkovitz  and  Gamkrelidze.   This  is 
done  by  introducing  new  multipliers   X(t)   such  that 

T       T  9C1 

pj(t)  -  Xj(t)  -  ^(t)  ^  , 

~  (187) 

T        T  2 

V(t)  -jg<t>  -y2(t)  jf  . 

It  is  well  known  that  we  may  then  make  the  identifications  (25) . 

As  should  be  clear  to  the  reader,  Problem  I  is  a  special  case  of 
Problem  III.   However,  the  development  of  necessary  conditions  of  optimality 
for  Problem  III  is  much  more  tedious  and  not  the  same  as  for  Problem  I. 
The  development  of  these  more  general  conditions  will  be  given  in  the 
future. 


It  should  again  be  recalled  that  this  statement  must  be  modified  for  an 
absorbing  state  boundary. 


A- 5  6 


8 .   Further  Extensions. 

There  are  two  directions  of  import  in  which  the  above  results  may 
be  extended: 

(a)  additional  inequality  constraints  on  the  strategies, 

(b)  higher  order  SVIC's. 

It  is  of  interest  to  consider  a  problem  (denoted  as  "Problem  IV") 
similar  to  Problem  or  Problem  II  in  which  there  are  additional  inequality 
constraints  on  the  (vector)  strategies  U  and  V. 


Problem  IV. 

maximize  minimize  {G(T ,w(T) ,x(T) ,y (T) )  + 

U        V  I 


L(t,w,x,y,u,v)dt} 


****  *"w  r*s 


0 


subject  to:         w  =  f  (t  ,w,x,y  ,v)  , 

x  =  g(t,w,x,y,u,v) , 
y  =  h(t,w,x,y,u) , 


<w   <~ 


u(t)   is   p-vector  of  strategic  variables  which  satisfies 
u(t)  £  ftU(t,w,x,y)   for  all   t  €  [0,T], 

v(t)   is   q-vector  of  strategi  variables  which  satisfies 
v(t)  €  ftV(t,w,x,y)   for  all   t  6  [0,T], 

w(t)  *  a(t)   for  all   t  €  [0,T]        (scalar  ^^   variable 
y(t)  *  3(t)   for  all   t  €  [0,T]         inequality  constraints), 

with  scalar  terminal  condition  F(T,w(T) ,x(T) ,y (T) )  =  0, 


where 


ft  =  {u   K(t,w,x,y,u)  £  0} 
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and 

flV  =  {v  I  R(t,w,x,y,v)  ^  0}. 

It  is  a  straightforward  matter  to  combine  the  previous  development 
of  Berkovitz  [3]  with  our  development  for  Problem  I  to  develop  necessary 
conditions  of  optimality  for  Problem  IV.   Our  applications  of  the  results 
of  this  appendix  will  be  to  Lanchester-type  differential  games  of  this 
form  (see  equations  (1)  above) . 

Another  extension  of  import  is  the  extension  of  our  results  on 
differential  games  with  SVIC's  to  problems  with  higher  order  SVIC's. 
Similar  one-sided  optimization  problems  have  arisen  in  our  study  of 
Lanchester-type  optimal  control  problems  [26]  (see  also  [23]).   Therefore, 
we  consider  the  following  problem  (denoted  as  "Problem  V") . 


Problem  V. 


maximize  minimize  {G(T,x(T)  ,y  (T)  )  + 

U        V 


***  r*s 


L(t,x,y,u,v)dt}, 


0 


subject  to:   x,  =  fn(t,x1,x0), 
#vi   ~l_   ~i  ~z 


where 


52  =  &(t»5i'52»XiL'X2«v)« 

£2  =  fc(t»5i»52'Xi«&»u)» 

u(t),  v(t)   are  unrestricted  (scalar)  strategic  variables, 
C^t,^)  *  0  for  all  t  €  [0.T]     (gcalar  state  variable 
C  (t,y_  )  ^  0   for  all  t  6  [0,T]     inequality  constraints), 
with  scalar  terminal  condition  F(T,x(T) ,y (T) )  =  0, 

— — — 0-*,  +•*, 


T    ,  T  T\     .   T    t   T  Tv 
x  =  (xx,x2)   and  L     =  (Ll>L2) 
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We  make  the  following  assumptions  about  C  (t,xn)   in  Problem  V: 

k  k      k 

(1)  (C.)  =   0   for   k  =  l,...,p-l   where   (C.  )  =  ^~  , 

1  V  1  dt 

(2)  the  first  derivative  of   C.  (t  ,x..  (t) )   which  explicitly  contains 

th 
the  strategic  variable  v  in  the  p— -  and   (C. )   ^  0  along 

an  optimal  trajectory, 


(3)   (C  )   =  0   for  k  =  1,2,...   (i.e.  for  all  integers  k  £  1) . 

Similar  assumptions  are  made  about  C0(t,y,)..   Other  assumptions  that  are 

Z        ~1 

required  in  the  development  of  necessary  conditions  are  similar  to  those 
made  in  previous  sections.   The  results  of  Jacobson,  Lele,  and  Speyer 
[12]  are  key  to  establishing  first  order  necessary  conditions  of  optimality 
for  Problem  V. 
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APPENDIX  B:   A  Differential  Game  Model  for  the  Study  of  Optimal  Air-War 
Strategies . 

1 .   Introduction. 

An  important  question  for  defense  planners  is  what  are  appropriate 
missions  over  the  course  of  a  tactical  campaign  for  tactical  air  power. 
The  answer  to  this  question  has  far  reaching  implications  for  Navy  air 
forces  (both  carrier-based  and  land-based)  (and,  of  course,  the  Air  Force). 
The  quest  for  an  answer  to  this  type  of  question  has  been  in  large  part 
the  genesis  of  modern  quantitative  methods  of  military  analysis.   One  has 
only  to  recall  the  classical  work  of  Lanchester  [10]  or  that  reported  in 
Morse  and  Kimball  [11]  (see  especially  pp.  73-77).   More  recently,  the 
USAF  Studies  and  Analysis  Group  has  been  using  quantitative  methodology 
[24]  in  trying  to  answer  such  questions. 

There  are  many  analysis  approaches  to  answering  the  question  of 
the  "best"role  (over  time)  of  tactical  air  support.   These  range  from 
operational  gaming  (see  [12]  or  [23]  for  discussion  of  terminology  and 
background)  to  analytical  solution  of  an  idealized  differential  game. 
However,  as  pointed  out  by  L.  Berkovitz  and  M.  Dresher  (see  p.  612  of  [1]), 
"operational  gaming  is  not  a  helpful  device  for  solving  a  game  or  getting 
significant  information  about  the  solution."   We  believe  that  there  is 
much  to  be  gained  by  a  more  analytical  study  of  an  idealization  of  the 
basic  problem  (see  also  [18]  or  [19]).   Therefore,  we  will  consider  a 
simplified  differential  game  model  for  the  "best"  (in  the  sense  discussed 
in  Section  below)  use  of  air  power,  with  our  eye  on  Navy  applications. 

As  we  have  noted  in  Appendix  A  and  elsewhere  [18],  [19],  and  impor- 
tant gap  in  the  previous  theory  of  differential  games  has  been  the  lack 
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of  multiplier  conditions  for  differential  games  with  state  variable 
inequality  constraints  (SVIC's).   It  should  be  noted  (see  also  [19],  [21]) 
that  state  variable  inequality  constraints  are  present  in  all  Lanchester- 
type  dynamic  tactical  allocation  problems  because  force  levels  (which  are 
represented  by  state  variables)  are  required  to  be  non-negative  (or  some 
equivalent  condition).   Thus,  the  theoretical  state-of-the-art  previously 
was  not  adequate  to  solve  all  such  problems.   Hence,  this  (our  study  of 
the  below  problem)  is  an  important  application  of  our  new  theoretical 
results  given  in  Appendix  A.   It  should  be  pointed  out  that  the  current 
state-of-the-art  in  optimal  control  theory  (at  least  its  application  in 
operations  research)  has  not  even  been  sufficient  to  allow  routine  solution 
of  Lanchester-type  optimal  control  problems  (see  [19])  because  of  the  pres- 
ence of  SVIC's  (see  [17],  [22]). 

In  this  appendix  then  we  will  consider  an  idealization  of  a  tactical 
air-war  game.   Versions  of  this  problem  have  been  considered  by  a  number 
of  workers  [1],  [2],  [5],  [6],  and  [7].   A  particularly  thorough  discussion 
of  modelling  aspects  of  the  problem  is  to  be  found  in  [6] .   This  is  the 
best  reference  on  modelling  aspects  that  the  author  has  encountered  so  far 
and  is  the  source  of  many  ideas.   A  special  version  of  the  problem  that 
we  will  consider  has  been  studied  by  R.  Isaacs  (see  pp.  96-104  in  [7]). 
Although  he  obtains  a  correct  and  thorough  solution  to  the  problem  he 
considered,  Isaacs  does  not  employ  the  equivalent  of  optimality  conditions 
for  differential  games  with  SVIC's.   The  optimality  of  constrained  subarcs 
is  not  examined,  and  Isaacs'  approach  may  not  yield  optimal  strategies 
for  more  general  problems. 
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A.  Friedman  (see  pp.  239-240  of  [4])  also  considers  Isaacs'  "War 
of  Attrition  and  Attack."   His  treatment  is  not  nearly  as  thorough  as  that 
of  Isaacs,  and  Friedman's  solution  is  not  complete.   He  (as  does  Sternberg 
[16])  misses  certain  subtle  aspects  first  observed  by  Isaacs  (see  pp.  102- 
104  of  [7]  and  also  pp.  948-949  of  [9]).   Moreover,  the  emphasis  of 
Friedman's  research  [4]  is  on  the  existence  of  value  for  differential 
games  with  SVIC ' s .   The  analogues  of  the  well-known  control  theory  multi- 
plier conditions  for  constrained  subarcs  [8]  are  not  developed.   In  fact, 
the  treatment  of  the  example  on  pp.  239-240  of  [4]  by  Friedman  is  inade- 
quate, since  his  approach  fails  to  yield  optimal  strategies  for  minor 
modifications  in  the  problem's  formulation  (see  below).   Thus,  in  this 
appendix  we  present  a  different  and  more  general  analysis  than  has  appeared 
in  the  literature  previously. 

The  problem  that  we  study  in  this  appendix  is  quite  significant 
from  the  standpoint  of  military  operations  analysis.   Currently,  the  USAF 
is  using  a  similar  (although  much  more  detailed)  model  (TAC  CONTENDER)  [24], 
which  has  been  widely  used  at  the  DOD/JCS  planning  level.   Our  research 
should  shed  light  on  cause-effect  relationships  between  modelling  assump- 
tions and  optimal  air-war  strategies  obtained  from  TAC  CONTENDER. 

One  may  consider  that  there  are  four  essential  parts  of  any  dynamical 
combat  optimization  problem: 

(1)  the  decision  criteria  (for  both  combatants) , 

(2)  the  model  of  conflict  termination  conditions  (and/or  unit  break- 
points) , 


(3)  the  model  of  combat  dynamics, 

(4)  the  decision  latitude  of  the  combatants, 
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By  decision  latitude  we  mean  the  extent  of  choices  that  decision  makers 
on  both  sides  can  actually  control.   The  other  aspects  have  been  discussed 
in  [19].   Let  us  mention  here  that  recently  Pugh  and  Mayberry  [15]  have 
discussed  the  first  topic  (measures  of  effectiveness)  (see  also  [14]). 
They  also  propose  a  computational  scheme  [14]  (Lagrange  dynamic  programming 
[13]).   Our  analytical  work  here  (and  in  the  future)  provides  test  cases 
for  the  adequacy  of  their  computational  approach. 

Our  analysis  approach  is  more  idealized  than  the  TAC  CONTENDER  model 
or  the  conceptual  approach  of  Pugh  and  Mayberry  [14].   However,  we  feel 
that  our  research  complements  these  other  efforts.   We  view  that  our 
differential  game  analysis  has  several  significant  uses: 

(1)  provide  guidance  for  higher-resolution  studies, 

(2)  identify  cause-effect  relationships  between  optimal  tactics  and 
modelling  assumptions. 

2.   The  Model:   A  Generalization  of  the  Tactical  Air-War  Game. 

We  consider  an  idealized  model  in  order  to  gain  insights  into  optimal 
air-war  tactics.   Background  material  on  this  model  can  be  found  in  [6] 
(or  [1]).   The  problem  that  we  consider  is  a  generalization  of  the  tactical 
air-war  game  studied  by  R.  Isaacs  [7]  (i.e.  his  "War  of  Attrition  and 
Attack").   The  origin  of  this  problem  is  apparently  due  to  A.  Mengel  (see 
[6]). 

Consider  a  war  between  X  and  Y   forces.   A  land  war  takes  place 
and  its  location  will  be  denoted  as  the  FEBA  (Forward  Edge  of  the  Rattle 
Area) .   Both  the  X  and  Y   forces  have  a  single  type  of  aircraft  that 
can  fly  two  types  of  missions:   (1)  ground  support  which  means  flying  fire 
support  missions  for  the  ground  forces  to  influence  the  outcome  of  the  land 
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war  and  (2)  counter-air  which  results  in  the  shooting  down  of  enemy  planes 
(but  no  direct  help  for  the  ground  forces) .   The  problem  for  each  commander 
is  to  find  the  "best"  use  of  his  air  power  (x(t)   denotes  the  number  of   X 
aircraft) .   The  decision  making  criterion  for  each  commander  is  the  net 
amount  of  support  for  the  FEBA  measured  in  terms  of  the  "value"  of  total 
missions  flown  (or,  equivalently ,  weighted  tons  of  ordnance  dropped  or 
weighted  total  ground  support-aircraft  days).   For  planning  purposes,  the 
air  war  lasts  for  a  prescribed  duration  of  time,  denoted  as   T,   and  residual 
values  of  surviving  aircraft  (based  on  linear  utility  for  both  sides1  sur- 
viving  aircraft)  is  considered.    It  is  assumed  that  new  aircraft  are 
introduced  on  both  sides  at  constant  rates.   This  situation  is  shown  dia- 
grammatically  in  Figure  1. 


x(t) 


y(t) 


Figure  1.   Diagram  of  Generalization  of  Tactical  Air-War  Game 


I  am  indebted  to  MAJ  Ron  Kronz,  USAF  for  pointing  out  that  this  is  commonly 
done  in  similar  USAF  analyses  (i.e.  TAC  CONTENDER). 
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Mathematically  the  problem  may  be  stated  as 

t_ 


maximize  minimize  {v  x(t_)-v  y(t^)  +     [B (t) ux-A(t) vy]dt} , 

U        V      X    i    Y    i     j 

0 

with  stopping  rule:       tf  -  T  =  0, 

dx 
subject   to:  d7  =   r   ~    (l~v)a(t)y,  (1) 

(air-battle  dynamics) 

||  =   s   -    (l-u)b(t)x, 

with  initial  conditions   x(t=0)  =  x   and   y(t=0)  =  y  , 
and 

x,y  ^  0  (State  Variable  Inequality  Constraints), 

0  ^  u,v  ^  1      (Strategic  Variable  Inequality  Constraints), 

where 

x(t)   and   y(t)   are  the  numbers  of   X  and  Y   aircraft,  respectively, 
at  t ime   t , 

a(t)   and  b(t)   are  variable  (Lanchester)  attrition-rate  coefficients, 

r  and   s  are  replacement  rates, 

v  (v  )   is  the  value  of  one  surviving   X(Y)   aircraft, 

X   Y 

A(t)   (B(t))   is  the  value  of  one  Y (X)   aircraft  flying  ground 
support  for  a  unit  of  time  at  time   t, 

u(v)   is  the  fraction  of  total   X(Y)   aircraft  which  fly  ground 
support  missions. 

A  discussion  of  the  circumstances  under  which  the  above  air-war  equations 
have  been  hypothesized  to  apply  can  be  found  in  [20].   It  should  be  noted 
that  capital  letters  are  used  to  denote  (closed-loop)  strategies,  i.e. 
U  =  U(t,x,y)   and  V  =  V(t,x,y),   whereas  lower  case  letters  are  used  to 
denote  the  strategic  variables  which  are  their  outcomes  (or  realizations 
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of  the  strategies),  i.e.   u  =  u(t)  =  U(t,x,y).   Optimal  (and  also  extremal  ) 
strategies  will  be  denoted  as  follows:   U   and  V  . 


3.   Development  of  Basic  Necessary  Conditions  of  Optimality. 

It  should  be  clear  that  we  have  in  (1)  above  that  v v,v  ^  0,   that 
r,s  >  0,   and  that   a(t) ,b (t) ,A(t) ,B (t)  >  0   for   t  £  [0,T].   Friedman  has 
shown  that  the  differential  game  (1)  has  value  (see  pp.  231-232  of  [4]), 
and  it  may  be  shown  that  a  saddle  point  exists  in  pure  strategies  (see  pp. 
234-235  of  [4]).   We  now  will  develop  necessary  conditions  of  optimality 
for  (1). 

Applying  the  results  of  Appendix  A,  we  have  that  the  Hamiltonian  is 
given  by 

H(t,x,y,p,q,u,n,u,v)  =  B(t)xu  -  A(t)yv  + 

p{r-(l-v)a(t)y}  +  q{s-(l-u)b(t)x}  -  u(t)x  +  n(t)y,     (2) 


where 


and 


!=  0  for  x  >  0, 

^  0  for  x  =  0, 

!=  0  for  y  >  0, 

;&  0  for  y  =  0. 


We  have  adopted  above  the  following  correspondence  between  state  and  adjoint 
variables : 

state  variable  dual  variable 

x  P, 

y  q. 


4. 

We  use  the  term  extremal  to  denote  a  trajectory  on  which  the  necessary  condi- 
tions of  optimality  (see  Appendix  A)  are  satisfied  at  every  point  in  time. 


It  is  also  convenient  to  define 


$(tf  ,x(tf),y(tf),a)  =  vxx(tf)  "  vyy(tf)  -  a(tf-T).  (3) 

The  adjoint  system  of  differential  equations  for  the  dual  variables  is 

|£  =  _  M  =  _  B(t)u*  +  (l-u*)b(t)q  +  u,  (4) 

at     ox 

j*  =  _  |S  =  A(t)v*  +  (l-v*)a(t-)p  -  n.  (5) 

The  boundary  conditions  at   t  =  tf   for  the  adjoint  variables  are 

=  0   for   x(t  )  >  0, 
p(t=tf)  =  vx  -  v   where  v  1  (6) 

'  ;>  0   for   x(t  )  =  0, 

.    =  0   for   y(tf)  >  0, 
q(t=tf)  =  -vy  +  \>2     where  v2  1  (7) 

*  ;>  0   for  y(t  )  =  0. 

We  also  have  the  transversality  condition 

H(tf)+|£-=0, 

which  yields  that  a   =  H(t  )  .   This  latter  condition,  however,  is  not  useful 
for  determining  optimal  strategies. 

When  x,y  >  0,   the  extremal  strategic-variable  pair,  denoted  as 
(u* ,v*) ,   is  determined  by  the  max-min  principle.   Hence,  we  consider 

maximize  minimize  H(t ,x,y ,p,q,u=0,n=0,u,v) ,  (8) 

O^u^l    0£v:sl 

which  by  (2)  yields 


maximize  xu{B(t)  +  b(t)q},  (9) 

0<ius;l 
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and 


minimize  y(-v){A(t)  -  a(t)p}.  (10) 


Thus  for  x,y  >  0,   we  have 


1   for   (-q(t))  <  |^-, 


u*(t)  =  <  (11) 

*  0   for   (-q(t))  >  f^j, 


and 


v*(t)  =  {  (12) 

0   for  p(t)  >  a(t). 

We  must  further  investigate  the  possibility  of  singular  subarcs  (see 

SH 
Chapter  8  of  [3])  on  which,  for  example,  —  =  0   for  a  finite  interval 

ou 

of  time  (so  that  all  its  time  derivatives  vanish) .   For  a  u-singular 

3H 
subarc,  the  condition  that  -r—   =  0  with  x,y  >  0  yields  that  on  a  u- 

du 

singular  subarc  we  must  have 

q(t)=-fM  (13) 

The  condition  —  f-r— )  =  0  along  with  (12)  then  yields 
dt  <*u 

A(t)v*  +  (l-v*)a(t)p(t)  =  -^{f{7yl»  (^) 

or,  equivalently ,  by  (12) 

maximum  {A(t)  ,a(t)p(t)  }  =  -  — { -^y }  .  (15) 

Considering  the  adjoint  equations,  it  is  unlikely  that  (15)  would  hold  for 
a  finite  interval  of  time.  Moreover,  when  a(t) ,  b(t)  ,  A(t) ,  and  B(t) 
are  all  constants  (e.g.  a(t)  =  a  =  constant),  it  is  immediately  seen  that 
(15)  cannot  hold,  since 
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maximum  {A,ap(t)}  £  A  >  0  =  -  -r-{—}' 

b 

Thus,  it  is  impossible  (in  general)  to  have  a   u-singular  subarc  of  an 

optimal  trajectory  for  (1).   In  a  similar  fashion,  it  is  readily  shown 

that  it  is  impossible  (in  general)  to  have  a  v-singular  subarc. 

3.1.   Necessary  Conditions  of  Optimality  on  Constrained  Subarc  for  x. 

X        X 
On  a  constrained  subarc  on  which   x(t)  =  0   for   t   £  t  £  t„   the 

e        it 

dx  X        X 

Y   strategic  variable  v*(t)   is  determined  by  —  =  0   for   t   <  t  <  t. 

and  hence 

(1-v*)    = 


a(t)y(t)    ' 
or 

v*(t)  - x  -  imm  for  ce  K c  *  il-  <i6> 

dx   X-  X- 

Since  we  must  have  0  £  v*  ^  1  and  -r—   (t   )  <  0   (here   t    denotes  a 

dfc   e  J  e 

left-hand  limit) ,  such  a  constrained  subarc  can  occur  only  when 

r  £  a(t)y(t)   for  t  <  t  <;  t£,  (17) 

XX  XX 

with   r  <  a(t  )y(t  ).   Since   x(t)  =  0   for   t   £  t  £  t„  ,   the  dimension 
e    e  e        £ 

of  the  state  space  may  be  thought  of  as  being  reduced  by  one,  and  hence 

the  dimension  of  the  adjoint  variable  space  is  reduced  by  one  through 

—  =  0.   This  yields 
3v 

/  x    A(t)        X        X  /10, 

P(t)  =  I(I)   f°r   V*  *  *  V  (18) 

The  multiplier  u(t)   is  determined  by  -r-[ir- )    =  0  and  hence 
v  J      dt^3vJ 

u(t)  =  B(t)u*  +  (l-u*)b(t)(-q(t))  +  ^7(^777}   for   tX  <  t  <  t*    (19) 

dt  La (t)         e        a 
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Thus,  the  necessary  condition  of  optimality  u(t)  ^  0   leads  to  that  on 

a  constrained  subarc  with  x(t)  =  0   for  a  finite  interval  of  time  we  must 

have 

B(t)u*  +  (l-u*)b(t)(-q(t))  £  --IjMtl}   for   tX  <  t  <  t*      (20) 

dt la (t) '  e        £ 

X         X 
When  y  >  0  and  x(t)  =  0   for   t  £  t  £.   t„  ,   we  have  that  v*(t)   is 

e        I 

dx 
determined  by  the  condition  that  -^—  =   0.      It  should  be  noted  that  (18) 

at 

implies  that  (10)  is  identically  satisfied.   In  this  case  only  u*   is  to 
be  determined  by  the  max-min  principle  (8).   However,  when  x(t)  =  0,   then 
the  max-min  principle  (or,  equivalently ,  (9))  no  longer  yields  that  we  must 
have  u*(t)   given  by  (11).   However,  we  may  still  take  (11)  to  hold  by  a 
continuity  argument  in  which  we  consider  x(t)  =  €  >  0  and  then  let   €  ■+  0. 
Thus,  (11)  may  be  used  to  write  (20)  as 

maximum  {B  (t)  ,b  (t)  (-q(t) )  }  ^  -  -^77^   for   tX  <  t  <  t*      (21) 

dt  a(t;         e        Jc 

Remark :   The  significance  of  (20)  is  that  the  optimality  of  Y  pursuing  a 
strategy  which  results  in  x(t)  =  0   for  a  finite  interval  of  time 
depends  upon  what  X's   optimal  strategy  for  using  the  aircraft 
would  be  if  he  had  them.   This  type  of  behavior  in  which  the 
optimality  of  one  player  steering  the  system  along  a  state-constrained 
subarc  depends  upon  the  other  player's  optimal  strategy  is  unique 
to  differential  games.   Furthermore,  it  apparently  has  not  been 
noted  in  the  literature  previously. 

It  remains  to  discuss  the  corner  conditions  which  must  hold  at  entrances 

to  and  exits  from  constrained  subarcs  on  which  x(t)  =  0   for  a  finite  interval 

X        X 
of  time.   At  entrance  to  constrained  subarc  with  x(t)  =  0   for   t  £  t  £  t' 

e        Jt 

(see  Figure  2  below) ,  we  must  have 
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x=0 


Figure  2.   Entry  to  and  Exit  from  Constrained  Subarc 


(A   \  /   AT.  f       Av 

te  }  =  P(te  }  =  P(te)} 


q(te  )  =  q(te  )  =  q(te), 


(22) 


H(tf)  =H(tf), 

where  H(t*~)   denotes  H(t*  x(  A=0  ,y  ( A  ,p(t*~)  ,q(t*~)  ,u=0  ,n=0 ,u*  (t*~)  , 

X— 
v*(t   )=1) .   It  is  readily  seen  that 

H(t*~)  =  p(t*){r  -  a(t*)y(t*)}  +  q(t*)s, 


and  that 
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H(tf)  =  -A(tX)y(tX)U  -      r     >  +  q(tX)s, 

a(te)y(te) 

A(tX) 

so  that  H(tX~)  =  H(tX+)  ~  p(tX){r  -  a(tX)y(tX)}  =  f-  {r  -  a(tX)y(tX)}, 

e        e        e         e    e     a((.A)         e    e 

whence 

X    A(£e> 

P(te>  "  —t   •  «» 

a(t  ) 
e 

dx  X- 
since  we  must  have  -r~(t   )  <  0.   We  thus  see  that  the  corner  conditions 

at   e 

(22)  are  automatically  satisfied  when  (16)  through  (20)  hold,  and  thus  they 

yield  no  new  information. 

X        X 
At  exit  from  constrained  subarc  with  x(t)  =  0  for   t  £  t  £  t  , 

c  JC 


we  must  have 


tl    )    =  P(t£  }  =  p(V' 
q(tX")  =  q(tX+)  =  q(tX),  (24) 

H(tX")  -H(tf). 


It  is  readily  seen  that 


and  that 


H(tX  )  =  -A(tX)y(tX)U xr     >  +  q(tX)s, 

a(t1)y(t£) 


H(tX+)  =  -A(tX)y(tX)  +  (tX)r  +  q(tX)s, 
so  that  H(t£  )  =  H(tJ  )  «* 

X   A(t*> 
a(tA) 

Again,  we  see  that  the  corner  conditions  (24)  are  automatically  satisfied 

when  (16)  through  (20)  hold,  and  thus  they  yield  no  new  information. 
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3.2.   Necessary  Conditions  of  Optimality  on  Constrained  Subarc  for  y. 

Similar  analysis  yields  for  a  constrained  subarc  on  which  y(t)  =  0 

Y        Y 
for   t   £  t  £'t„   that  we  must  have 
e       a 

u*(t>  • x  -  barky  £or  tYeKt<  v  (26) 

Y        Y  Y    Y 

with   s  £  b(t)y(t)   for   t   <  t  £  t   and   s  <  b(t  )y(t  ).   Additionally, 


we  must  have 


(-q(t))  -  1^"  for  tY&   £  t"*  t\t  (27) 

and 

n(t)  =  A(t)v*  +  (l-v*)a(t)p(t)  +  ^If^yl  ^0   for   t*  <  t  <  t*.  (28) 

The  latter  condition  may  also  be  written  as 

A(t)v*  +  (l-v*)a(t)p(t)  :>  -  ^{f^}   for   t*  <  t  <  t*,      (29) 

or  by  use  of  (12)  (which  was  developed  from  (10)  for  y  >  0) 

maximum  {A(t)  ,a(t)p (t)  }  £  -  — -{~^}      for   tY  <  t  <  t] .      (30) 

at  b  {t)  e        J6 

The  corner  conditions  yield  that  at  entry  to  constrained  subarc  on 

Y  Y 

which  y(t)  =  0   for   t  £  t  £  t. ,   we  must  have 

,  Y-,     ,  Y+N     ,  YN 
p(te  )  -  p(te  )  =  p(te), 

1^1")  =  q(tI+)  =  q(tI}'  (31) 

(-q(tg)) f-  • 

Also,  the  corner  conditions  yield  that  at  exit  from  constrained  subarc  on 

Y  Y 

which  y(t)  =  0   for   t  £  t  £  t  ,   we  must  have 
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.   Y-,     ,  Y+N     ,  Y. 
p(tA  )  -p(t£  )  =p(t£), 

q(t£_)  =  q(tI+)  =  q(tI}'  (32) 

Y      B(tI} 

(-q(tp) f-  . 

b(t*) 


3.3.   Discussion  of  Optimality  Conditions  for  Constrained  Subarcs. 

In  this  section  we  will  discuss  some  implications  of  the  above 
necessary  conditions  of  optimality  that  must  hold  on  constrained  subarcs. 
Let  us  first  consider  the  case  in  which  the  following  hold: 

a(t)  =  a  =  constant, 

b(t)  =  b  =  constant, 

A(t)  =  A  =  constant, 

B(t)  -   B  =  constant. 

In  this  case,  the  necessary  condition  of  optimality  (21)  which  must  hold 

X        X 
on  constrained  subarcs  on  which  x(t)  =  0  for   t  £  t  £  t   with  t   <  t 

e        x.         e    jo 

is  automatically  satisfied,  since  we  have 

maximum  {B,b(-q(t))}  £  B  >  0  =  -  J^{z\  • 

a 

Similarly,  (29)  (or  equivalently  (30))  is  automatically  satisfied  on  con- 

Y        Y         Y    Y 
strained  subarcs  on  which  y(t)  =0   for   t  £  t  £  t   with   tfi  <  t£.   Thus, 

* 
we  see  that  necessary  conditions  of  optimality  are  always  satisfied  for 

the  holding  of  enemy  aircraft  at  a  zero  level  for  a  finite  interval  of  time 

in  the  case  of  constant  coefficients  (i.e.  the  functions   a(t),   b(t),   A(t), 


At  least  the  ones  that  we  have  considered  above.   There  are  also,  of  course, 
conditions  related  to  the  transversality  conditions  (6)  and  (7) .   These  are 
examined  in  Section  4  below. 
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and   B(t)   are  all  constant)  in  (1).   This  is  precisely  the  case  of  (1) 
previously  studied  by  others  [4],  [7],  [16].   In  the  case  of  constant 
coefficients,  there  is  no  question  about  the  optimality  of  a  policy  which 
results  in  the  annihilation  of  all  enemy  aircraft  during  the  appropriate 
phase  of  the  air  war  (see  Section  5  below).   However,  in  the  case  of  time- 
dependent  returns  from  ground  support  and  time-dependent  Lanchester  attri- 
tion-rate coefficients,  the  optimality  of  such  a  policy  is  not  always 
guaranteed.   Furthermore,  the  optimality  of  a  strategy  which  results  in 
the  annihilation  of  enemy  aircraft  depends  upon  the  enemy's  strategy  for 
using  his  aircraft  if  he  did  have  them  (see,  for  example,  (20)  above). 
More  precisely,  for  example,  the  multiplier  condition  u(t)  ^  0   (upon 
which  Y  bases  his  strategy)  which  must  hold  on  a  constrained  subarc  on 
which   x(t)  =  0   depends  upon  X's   optimal  strategy  (i.e.  the  enemy's 
strategy) .   This  behavior  is  unique  to  differential  games  and  has  apparently 
never  been  noticed  before. 

Let  us  now  give  two  simple  examples  which  show  the  complexity  in 
the  general  case  of  determining  whether  X  should  try  to  shoot  down  all 
enemy  planes  (during  the  appropriate  phase  of  the  air  war).   Thus,  we 

consider  the  necessary  condition  of  optimality  (29)  which  must  hold  on  a 

Y        Y         Y    Y 
constrained  subarc  on  which  y(t)  =  0   for   t   £  t  £  t„   with   t   <  t„  . 

J  e        I  e    I 

In  the  first  case,  let  us  assume  that  the  following  hold: 

a(t)  =  a  =  constant, 

b(t)  =  b  =  constant. 

Then,  (30)  takes  the  form 

maximum  (A(t),ap(t))  ^  --j-  . 

b 
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Let  us  assume  that  we  have  A(t)  ^  ap(t) .   Then  we  have  for  the  appropriate 
phase  of  the  air  war 

(a)  if  -7—^0,   X   should  try  to  shoot  down  all  the  Y   planes, 

dR 

(b)  if  -r—  <  0,   X  may  want  to  fly  all  ground  support  missions  now 

and  shoot  down  enemy  planes  later  in  this  case  in  which  X's   rate 
of  return  from  ground  support  is  decreasing. 
In  the  second  case,  let  us  assume  that  the  following  hold: 

A(t)  =  A  =  constant, 

B(t)  =  B  =  constant. 

Then,  (30)  takes  the  form  that  we  must  have 

maximum  (A,a(t)p(t))  :>  r^rt\\2   j^  » 

in  order  for  it  to  be  optimal  to  have  y(t)  =  0  for  a  finite  interval  of 
time.   Then  we  have  for  the  appropriate  phase  of  the  air  war 

(a)  if  —  £  0,   X  should  try  to  shoot  down  all  the  Y  planes, 

(b)  if  —  >  0,   X  may  want  to  forget  about  shooting  down  Y  aircraft 

dt 

today  because  tomorrow  he  will  be  so  much  more  effective  doing  this. 


3.4.   Introduction  of  the  Switching  Functions   S  (t)   and   S  (t) . 
a u v 

It  is  convenient  to  consider  the  switching  functions   S  (t)   and 
S  (t)   defined  by 

Su(t)  =  (-q(t))b(t)  -  B(t),  (33) 

and 

Sv(t)  =  p(t)a(t)  -  A(t).  (34) 
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Then,  we  may  re-write  the  extremal  control  laws  (11)  and  (12)  for  the 
strategic  variables  as 
for  y  >  0: 


u*(t) 


1   for   S  (t)  <  0, 
u 


0   for  S  (t)  >  0,  (35) 


and 

for  x  >  0: 


v*(t)  = 


1   for   S  (t)  <  0, 


0   for  S  (t)  >  0.  (36) 


Since  we  will  develop  the  solution  to  (1)  by  working  backwards  from 
the  end  at   tf  =  T,   it  is  convenient  to  introduce  the  backwards  time   x 
defined  by 

t  =  T  -  t.  (37) 

It  should  be  noted  that  -r~  =  ,   *  i —  =  -  ,  .   We  have  then  from  (33)  and 

dt   dx    dx    dx 


(34) 


and 


Su(x)  =  (-q(x))b(x)  -  B(x),  (38) 


Sy(x)  =  p(x)a(x)  -  A(x),  (39) 


where  A(x)  =  A(T-x) ,   etc.  and   p(x)  =  p(T-x)   and  we  again  denote 
p(x)   as   p(x),   etc.   Then  from  (35) 
for  y  >  0: 


u*(x)  = 


1   for   S  (x)  <  0, 
u 


0   for   S  (x)  >  0,  (40) 
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and  it  follows  from  (38)  and  the  adjoint  equations  that 


d(t){A(t)v*  +  (l-v*)a(x)P(x)}  +  (-q(i)^-  "  ^  for   y  >  0, 

dS    (  T     T 

dx~  =  i 

0   for  y  =  0  (for  a  finite  interval  of  time) .            (41) 


Similarly, 

for  x  >  0: 


/  1   for  Sv(x)  <  0, 
v*(t)  = 

(  0   for   S  (t)  >  0, 


(42) 


and  it  follows  from  (39)  and  the  adjoint  equations  that 


a(x){B(x)u*  +  (l-u*)b(x)(-q(x))}  +  p(x)^  -  ^  for   x>0, 

v  =  I 

dx    ) 

0  for  x  =  0   (for  a  finite  interval  of  time) . 


4.   Extremal  End  States. . 

By  an  extremal  end  state  we  mean  a  terminal  point  in  the  state  space 
for  which  necessary  (transversality)  conditions  of  optimality  are  satisfied, 
Let  us  first  consider  in  detail  the  situation  for  the  X  aircraft.   There 
are  two  possibilities  which  we  denote  as  follows: 

(a)       E^  :  x(T)  >  0, 

and      (b)       E^  :  x(T)  =  0. 
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For   E  ,   there  are  two  further  subcases  as  follows: 

(1)  x(T)  =  0  with  x(t)  =0   for   tX  <:  t  <:  T  where   tX  <  T, 

(2)  x(T)  =  0  with  x(t)  >0   for   T  -  6  <;  t  <  T  where   6  >  0. 

Let  us  first  consider  the  end  state  E   :  x(T)  >  0.   By  (6)  at 

t,  =  T,   we  have  p(t=t  )  =  v  ,   since  x(t  =T)  >  0.   Thus, 
t  t      X  t 

for  x(tf)  >  0  :     p(t=tf)  =  vx.  (44) 

Also,  by  (36)  (or  equivalently  (12)) 

A(T) 


(  X   f°r  VX  *  a(T)> 
v*(t=t  )  =  { 

(  0   for  v„  *  A(T)- 


X  *  a(T).  (45) 


Comment:   If  v  =  0,   then  v*(t=t  )  =  1.   This  is  the  only  case  considered 

X  x 


in  previous  analyses  [4],  [7],  [16^ 


Let  us  now  consider  necessary  conditions  of  optimality  for  the  end 
state  E   :  x(T)  =  0.   As  above,  there  are  two  subcases  to  be  considered. 
In  case  (1):   x(T)  =  0  with  x(t)  =0   for   t^  <;  t  £  T  where   tX  <  T 
(i.e.  at  the  end  the  system  has  been  on  a  constrained  subarc  for  a  finite 
interval  of  time) ,  we  have  by  (18)  that 

P(T)  -  ±jg,       _  (46) 

since  the  system  has  been  on  a  constrained  subarc  for  a  finite  interval 
of  time.   Also,  by  the  transversality  condition  (6),  we  have  that 


p(T)  =  v  -  v   where  v  ^  0. 

X     _L  J_ 
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By  (46)  and  (47),  we  have  vn  = 


A(T) 


v pp-   ^  0,   so  that  we  must  have 


A(T) 
v  >   — - — — 
X  a  a(T) ' 


(48) 


for  it  to  be  optimal  to  have  x(T)  =  0  when  the  system  has  been  on  a 
constrained  subarc  for  a  finite  interval  of  time.   In  case  (2) :   x(T)  =  0 
with  x(t)  >  0   for  all   t  €  [T-6 ,T)   where  6  >  0,   it  is  clear  that  we 
must  have  v*(t)  =0    for   t  €  (T-6   T)  c  (T-6  ,T)   where   0  <  6   «£  6 , 
since  this  is  the  only  way  that  we  can  reach  x(T)  =  0  with  x(t)  >  0 

for  all   t  £  [T-6,T).   By  (10)  (see  also  (12))  in  order  to  have  v*(t)  =  0 

A(t) 
for  all   t  G  (T-6  ,T)   we  must  necessarily  have  p(t)  ^  — r-r--   By  the 
l  a  (,  t ) 

continuity  of   p(t)   (see  Appendix  A)  it  is  necessary  that  we  have 


P(T)  £ 


A(T) 
a(T) 


(49) 


We  also  have  the  transversality  condition  (47)  so  that   p(T)  =  v  -  v   k 

A      I 

A(T)/a(T)   which  we  may  also  write  as  v  ^  A(T)/a(T)  +  v  >  A(T)/a(T), 

X  1 

since  v  ^  0.   Thus,  we  see  that  (48)  is  again  a  necessary  condition  for 
it  to  be  optimal  to  have  x(T)  -  0  but  x(t)  >  0   for  all   t  fc  [T-6  ,T) 
where   6  >  0. 

Considering  the  above,  we  see  that  we  have  proved  the  following 
theorem. 


THEOREM  1: 

If  an  optimal 

strategy  for  Y   is  to  result 

in  x*(T)  =  0 

then  it  is  necessary  that 

A(T) 
VX^a(T)' 

t 


It  should  be  recalled  that  we  have  shown  it  to  be  impossible  to  have  a 
singular  solution  for   x,y  >  0.   Hence,   v*(t)   must  be  equal  to   0   or   1 
almost  everywhere  in  time. 
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In  other  words,  in  order  that  the  end  state  E   :  x(T)  =  0  be  an  extremal 
end  state  we  must  have  v  ^  A(T)/a(T).   We  also  have  the  following  as  an 

A. 

immediate  corollary. 


ACT) 
COROLLARY  1.1:   If  v  <  .    ' ,   then  an  optimal  strategy 

for  Y  leads  to  x*(T)  >  0. 


Remark:   In  the  problem  considered  by  Isaacs  [7]  and  others  [4],  [16] 

previously,  we  have  v  =  0,   A(t)  =  1,   and  a(t)  =  a  =  constant 

A. 

so  that  an  optimal  strategy  for  Y  resulted  in  x*(T)  >  0.   The 
optimality  of  this  condition  apparently  was  never  examined  pre- 
viously.  In  fact,  an  implicit  assumption  in  Friedman's  example 
on  pp.  239-240  of  [4]  is  that  an  optimal  policy  results  in 
x*(T)  >  0.   From  Theorem  1  we  see  that  this  need  not  be  true  in 
general . 

In  a  similar  fashion,  we  can  prove  the  following  theorem. 


THEOREM  2: 


If  an  optimal  strategy  for  X  is  to  result  in 
y*(T)  =  0,   then  it  is  necessary  that 

B(T) 


VY  *  b(T) 


We  also  have  the  following  as  an  immediate  corollary. 


COROLLARY  2.1: 

B  ( T) 
If  v..  <  ,  ,_, ,   then  an  optimal  strategy 
i    b  (.1; 

for  X  leads  to  y*(T)  >  0. 
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Remark:   It  seems  appropriate  to  point  out  the  following  interpretations: 

(a)  a(T)v  ■  rate  at  which  one  Y  aircarft  can  destroy  value 
of  X  aircraft  at  end  of  campaign, 

(b)  A(T)  =  rate  of  return  from  one  Y   aircraft  from  flying 
ground  support  missions  at  end. 

Considering  the  above,  the  meaning  of  Theorem  1  is  clear:   in 
order  for  Y   to  follow  a  strategy  which  results  in  x(T)  =  0, 
it  must  be  to  his  advantage  to  do  so. 

5.   Solution  for  Special  Case  of  Constant  Coefficients. 

In  this  section  we  will  develop  the  solution  to  (1)  for  the  special 
case  in  which  all  the  coefficients  are  constants.  In  other  words,  we  will 
consider  the  case  in  which  the  following  hold: 

a(t)  =  a  =  constant, 

b(t)  =  b  =  constant, 

A(t)  =  A  =  constant, 

B(t)  =  B  =  constant. 

For  notational  convenience  we  will  again  denote  a  as  a,   etc.   It  is 
then  convenient  to  re-state  the  problem  as  follows: 

t. 


"f 
t 


(Bux-Avy)dt}, 


maximize  minimize  {v  x(tf)-v  y(t  )  + 
U        V      X   t    Y    t 

with  stopping  rule:   tf  -  T  =  0, 

subject  to:       —  =  r  -  (l-v)ay,  (50) 

(air-battle  dynamics) 

&  =   s  -  (l-u)bx, 
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with  initial  conditions     x(t=0)  =  x_   and  y(t=0)  =  y0, 
and 

x,y  k  0  (State  Variable  Inequality  Constraints), 

0  £  u,v  £  1  (Strategic  Variable  Inequality  Constraints) , 

where  a  is  a  given  constant  and  similarly  for  b,   A,   and  B. 

Let  us  now  review  the  necessary  conditions  of  optimality  that  we 
have  developed  for  (50)  in  Section  3.   Let  us  recall  the  switching  functions 
in  backwards  time  t  =  T  -  t.   These  are 

Su(t)  =  b(-q(x))  -  B,  (51) 

and 

Sv(x)  =  ap(x)  -  A.  (52) 

We  had  developed  earlier  (see  (40)  through  (43))  the  following  character- 
ization of  optimal  strategies: 


for  y  >  0 


1  for  S  (t)  <  0, 


u 


u*(t)  = 


0  for  S  (x)  >  0,  (53) 


and 

for  x  >  0: 


v*(x)  = 


1  for  S  (x)  <  0, 


0  for  S  (x)  >  0,  (54) 


with 


dg     /  b{A  *  +  (l-v*)ap(x)}  >  0     for   y  >  0, 

ar-       V  (55) 

0     for  y  =  0    (for  a  finite  interval  of  time) , 
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and 


dS 
d 


j  a{Bu*  +  (l-u*)b(-q(r))}  >  0     for   x  >  0, 


(56) 


0 
dS 


for   x  =  0   (for  a  finite  interval  of  time) 


Thus  we  see  that  -—  >  0  with  strict  inequality  for   y  >  0   and  similarly 
for   sv(t).   From  (15)  we  see  that  singular  subarcs  are  impossible. 

On  a  constrained  subarc  on  which   x(t)  =0   for   t   <  t  <  t„      we 

e        x. 

have  shown  the  following  to  be  necessary  conditions  for  optimality: 


v*(t)  =  1 ~-     for   tX  <  t  <  t* 

ay ( t )        e        I 


r  <  ay(t) 


,     s  A 

p(t)  -- 


,  X        X 

for   t  <  t  <  tn , 
e        I 


X  ^      X 
for   t  <  t  <  t.  . 
e        H 


(57) 
(58) 
(59) 


The  requirement  (21)  is  automatically  satisfied,  since 


maximum  {B,b(-q(t))}  >  B  >  0  = 


drAi   _     X        2 

-  tH-}   for   t   <  t  <  t. 

dtlaJ        e        J 


Similar  consitions  hold  on  a  constrained  subarc  on  which  y(t)  =  0   for  a 
finite  interval  of  time. 

Concerning  extremal  end  states,  we  have  developed  the  following. 


PROPOSITION  1: 

If  an  optimal  strategy  for  Y   is  to  result 

in  x*(T)  =  0,   then  it  is  necessary  that 

A 
VX2a  • 

PROPOSITION  2:   If   v„  <  — ,   then  an  optimal  strategy  for  Y 

X   a 

leads  to   x*(T)  >  0. 
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PROPOSITION  3:   If  an  optimal  strategy  for  X   is  to  result 

in  y*(T)  =  0,   then  it  is  necessary  that 

B 

v   >  — . 
VY  -  b 


PROPOSITION  4:   If  vv  <  — ,   then  an  optimal  strategy  X 

i    b 

leads  to  y*(T)  >  0. 


We  also  have  the  following  boundary  conditions  for  the  adjoint  variables: 


p(t=T)  =  v  -  v   where  v 


=  0   for   x(T)  >  0, 
>  0   for   x(T)  -  0, 


(60) 


q(t=T)  =  vy  +  v   where   v 


=  0   for  y(T)  >  0, 
>  0   for  y(T)  =  0. 


(61) 


We  may  then  write  the  differential  equations  for  the  switching  functions 


as 


and 


dS     /  b{A  +  (1-v*)Sv(t)}     for   y  >  0, 

—  -  I 
dT     I 


(62) 


0     for  y  =  0   (for  a  finite  interval  of  time), 


dS     /  a{B  +  (1-u*)Su(t)}     for   x  >  0, 
~d~  =  ) 


(63) 


0     for  x  =  0   (for  a  finite  interval  of  time), 


with  initial  conditions 


S  (t=0)  =  (bv  -B)  -  bv.   and   S  (t=0)  =  (av  -A)  -  av 

U.  I  Z  V  A  .1 


(64) 
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5.1.   Synthesis  of  Extremal  Stratcgic-Variabl e  Pair . 
There  are  four  cases  to  be  considered: 

(1)  vv<l  (!)   vy<5 

(2)  vY  ,  I  (2)   vY  *  I 

These  cases  will  be  denoted  as  (II),  (12),  (III),  and  (112).   The  solution 
to  (50)  must  be  developed  for  each  case  separately.   It  should  be  noted 
that  Case  (12)  and  Case  (III)  are  symmetric:   Case  (12)   is  the  same  as 
Case  (III)  only  with  the  roles  of   X  and   Y   interchanged.   These  then 
are  three  cases  of  different  structures  for  the  optimal  strategies. 

In  each  case,  the  solution  to  (50)  is  developed  by  working  backwards 
from  the  end  of  the  war  at   t  =  t,  =  T   (or,  equivalently ,   T  =  0)   where 
the  boundary  information  (60)  through  (63)  is  known.   The  differential 
equations  for  the  switching  functions   S  (t)   and   S  (x)   may  then  be 
integrated  backwards  (using  (53)  and  (54)  (or  (57)  on  constrained  subarcs)) 
so  that  the  initial  conditions  are  just  met.   This  procedure  leads  to  syn- 
thesis of  an  extremal  strategic-variable  pair  and  a  corresponding  extremal 

* 

trajectory.    In  the  case  in  which  the  extremal  trajectory  is  unique  (in 

the  sense  that  there  is  a  single  extremal  path  through  any  point  of  the 
state  space  which  leads  to  the  terminal  manifold,  it  is  the  optimal  trajec- 
tory.  This  is  the  situation  in  all  cases  except  for  one  exceptional  set 
of  circumstances  for  Case  (12).   The  optimal  strategies  in  this  (exceptional) 
situation  have  not  been  resolved  at  the  time  of  the  writing  of  this  report. 
This  is  an  important  future  research  task,  since  the  development  of 


By  an  extremal  trajectory  we  mean  a  path  along  which  the  necessary  conditions 
of  optimality  (see  Appendix  A)  are  satisfied  everywhere  in  time. 
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computational  schemes  for  time-sequential  combat  games  (such  as  application 
of  Lagrange  dynamic  programming  as  proposed  in  [13],  [14])  depends  upon 
its  successful  completion. 

Case  (II):   v„  <  —  and   v„  <  --. 

__ — i 1  —  ■ V   ■    o ■  -  -    V — ■ — ■ — K 


Let  us  make  the  nonrestrictive  assumption  that 

A-av    B-bv 

—i r  *  ~w  •  (65) 

By  Propositions  2  and  4  we  have  that   x*(T)  >  0   and   y*(T)  >  0   so  that 
by  (64)  we  have 

S  (t=0)  =  bv  -  B  <  0   and   S  (t=0)  =  av   -  A  <  0.  (66) 

U  I  V  A 

One  obtains  from  (66)  via  (53)  and  (54)  that 

u*(t=0)  =  1  and  v*(x=0)  =  1.  (67) 

By  straightforward  continuity  arguments,  it  is  readily  seen  that 

u*(t)  -  1 


^  v*(t)  =  1 


for   t  e    [0,t  ],  (68) 


where   t1   denotes  the  "backwards  time"  of  the  first  switch  in  (extremal) 
tactics.   Using  (62),  (63),  and  (68),  we  see  that  the  differential  equations 
for  the  switching  functions  are  for   0  <  t  <  t 

dS 

-—  =   bA   with   S  (t=0)  =  bvv  -  B  <  0,  (69) 

dx  u  i 

and 

dS 

-r~^-  =   aB   with   S  (x=0)  =  av„  -  A  <  0,  (70) 

dT  V  A 
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so    that    for      0  <   x    <   T 


S    (t  )    =  bAr    +  bvv   -   B , 
u  Y 


(71) 


and 


S    (x)    =  aBx    +  av      -   A. 
v  X 


(72) 


The    "backwards"   switching    time      t ^      is   determined   by      S    (t=t    )    ■■■■   0      and 
hence 


u    u 


B-bv. 


T 


u     bA 


(73) 


Similarly,  we  have 


A-avx 

T   =  _ 

v     aB 


(74) 


The  assumption  (65)  then  yields  that  t,  =  t   <  t   so  that  v*(t)   switches 

1    v    u 

first  in  backwards  time  at 


A-av 


t,  =  t   = 


X 


1    v     aB 


(75) 


We  then  have  for  x(t)  >  0 


u*(t)  =  1 
v*(t)  =  0 


for   t  e  (t  ,t  ), 


(76) 


where   t9   denotes  the  "backwards  time"  of  the  second  switch  in  (extremal) 
tactics.   It  should  be  noted  that  by  (50)  and  (76)  we  have  y(x)  >  0   for 


0  <  t  <  t„.   Using  (62),  (63),  (71),  (72),  (75),  and  (76),  we  see  that  the 
differential  equations  for  the  switching  functions  are  for   x   <  t  <  t 


dS 


dx 


-   =  b{A+S  (t)}  with   S  (t=T.)  =  bAJ 
v  u    ]       l 


rA-av    ) 
X 

aB    j 

B-bv   \ 
bA 

}  <  0,      (77) 


and 
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dS 
dx 


aB    for    x  >  0 


for   x  =  0   (for  finite 
interval  of  time) 


with 


VT=V 


=  0 


(78) 


There  are  two  cases  to  be  considered 


(a)  x(t=T-T;L)  >  0, 

(b)  x(t=T-T])  =  0. 

In  Case  (a):   x(t=T-x  )  >  0,   we  have  that   x(t)  >  0   for   0  '  '  x  ^  T 
by  (50)  and  (76).   Hence,  (54)  and  (78)  yield  that  v*(t)  =  0   for  all 
x  >  T,  .   Integrating  (77)  and  (78),  we  obtain 


s  (T)=fk(BT+  )2_H(A)2+bA, 


2B 


2Bva; 


A-av, 
i 

~a¥ 


B-bv„i 


bA 


}. 


(79) 


and 


S  (t)  =  aB(x-x  )  =  aBx  -  (A-av  ) . 

V  J,  X. 


(80) 


The  switching  time   t   =  x„   is  determined  by   S  (t=t  )  =  0  and  hence 

u  2  u  u 


/  fB_bvv^     (A~aVY^ 

T9   =    T      =  \Uk2   +   2aABf   ~ iXl iT       "   avvl    >    T! 

2    u   aBlv         l  [  bA  J  {     aB  J '  XJ     1 


(81) 


Again,  (53)  and  (55)  yield  that  once  u*(t)  =  0   there  are  no  further 


switches  in  the  strategic  variables  as  we  progress  backwards  in  time. 

In  Case  (b):   x(t=T-x  )  =  0,   we  have  that  x(t)  =  0   for   x  »  x 

X  X       *  * 

<  t  <  t   and   x(t)  >  0   for   t   <  x  <   x_,   where   x~   denotes  the 

e  e        2  2 

"backwards  time"  of  the  second  switch  in  (extremal)  tactics  in  the  case 

X   i>  X 

in  which  x(t)  =  0   for   x   <  x  <  t   s  x  .   When   x(x)  =  0   for   x  =  x 

_L        e    z.  A*    x 

X   ^ 

<  x  <  x   <  x_,   we  obtain  from  (77)  and  (78)  that 

e    I 
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B-bv, 


fO-DVY1 


(82) 


and 


Sv(t)  =  0 


(83) 


X  X       'i/ 

When  x(T)  >  0   for   x   <  x  <  x9,   (77)  and  (78)  become  for   t   ^  x  ^  x„ 


dSu  x  x     rB_bv^ 

—  -b{A  +  8v(T)}  with   Su(T=T^)  -  bA{TX  -   -gJ 


]}>      (84) 


and 


dS 

-~  =  aB   with   S  (x=t  )  -  0 
dx  v    e 


(85) 


X       'V/ 
Integration  of  (84)  and  (85)  yields  that  for   x   <  x  £  x„ 

e        2 


S  (x)  =  ^r   (x-xX)2  +  bA(T-xX)  +  bA{TX  - 
u       2      e  e      l  e 


rB-bv, 


bA 


},       (86) 


and 


S  (x)  =  aB(x-x  ) 
v  e 


(87) 


The  switching  time   x   -  T*   is  determined  by   S  (x=x  )  =  0   and  hence 

u  2  u  u 


^2  -  *u  -  ^^(A^ivf^jf-M, 


(88) 


for 


k 


avx^  X      ^, 

~ir~\  =  t,  <  x    <  x_ 

aB  1  e  2 


fB-bvYl 


bA 


Thus,  we  have   x„  =  tJt  ).   It  is 
2    2  e 


readily  seen  that   x9   is  a  strictly  increasing  function  of   x   for 
/B-bv, 


X 
Tn  <  Te  < 


bA 


,   since  we  have 


3x. 


9x, 


=  1  - 


F- 


2aAB{ 


rB-bv, 


T   - 

e 


bA 


)> 


(89) 
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a. 

8T2  X 

It    is   readily   seen   that     — -  >  0      for      t      < 


rB-bv  ^  3x 

,      that     — -  =  0      for 


X         fB-bVYl 


bA 


3x 


3t, 


bA 


2  X 

,      and    that       — rr  <   0      for      t      > 

3t  e 


rB-bv 


maximum    (which  we   denote    as      x')    for      x      = 

2  e 

to   note   that 


rB-bv   - 
bA~, 


9x 


r-r—  I.      Hence,      x        has   a 


It   seems   appropriate 


i  f~  rB~bvv1 


rA-av  i 


l     aB    J 


}   -   avY)      for      x*  =    x  (go) 

X  el 


and 


x2  -   x2   = 


rB-bv, 


bA 


for 


X 
x      = 
e 


rB-bv, 


bA 


Thus,   we   see    that 


x„    <    x„    <   x'    = 


rB-bv, 


bA      ' 


(92) 


for 


fA-av  i 
x 


aB 


=   xn    <   x      < 
1  e 


rB-bv^ 


bA 


This  means    that   the   switching   time   for 


u*(t)      depends   upon  force   levels,    since   it  depends   upon  whether  or  not     x(t) 

rA-av, 


is  driven  to  zero  before   t=T-  t  =T- 


X 


aB 


when  T  >  x  .   Further- 


XX        X 
more,  when  x(t)  =  0   for  T-x  =t   <  t  <  t0  =  T  -  x, ,   the  switching 

e    e        *<        1 

time  for  X,   i.e.   x_  =  x  ,   depends  upon  the  time  (or,  equivalently ,  the 
backwards  time  x   )  at  which  the  X  forces  are  annihilated.   Again,  (53) 
and  (55)  yield  that  once   u*(t)  =  0   there  are  no  further  switches  in  the 
strategic  variables  as  we  progress  further  backwards  in  time.   This  situation 
is  shown  diagrammatically  in  Figure  3  below.   Thus,  we  see  that  X  switches 
to  concentration  on  ground  support  earlier  in  those  compaigns  in  which 


x  ,   y  ,   and  T  are  such  that  X  will  lose  all  his  planes  before 
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No t e :   Strategic  Variables 

X:   u 
Y:   v 


Figure  3.   Dependence  of   X   Switching  Time   t„  ~  t 
Upon  Force  Levels  (not  drawn  to  scale) . 


T  - 


A-av 


X 


>  0  but  after  T  - 
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*-bvY| 

— ~r— ]  >   0-    This  means  that   X  s   optimal 


strategy  is  to  use  his  planes  for  ground  support  earlier  in  such  cases  than 
when   x(t=T-x1)  >  0  because  they  are  annihilated  before  the  last  phase  of 
the  air  war  (when  both  sides  concentrate  on  ground  support).   Thus ,   X   uses 
his  planes  for  ground  support  while  they  are  still  available.   This  phenom- 
enon was  first  noted  (for  a  simpler  case  of  (50))  by  Professor  Rufus  Isaacs 
[7  J  (see  also  pp.  948-949  of  [9]).'' 

Case  (12):   vv  <  A-  and  v  >J^. 
a a — Y b 

By  Proposition  2  we  have  that   x*(T)  >  0   so  that  by  (64)  we  have 


S  (t=0)  =  av  -  A  <  0, 

V  A 


(93) 


and  by  (54)  we  obtain 


v*(t=0)  =  1, 


(94) 


By  Proposition  3  an  optimal  strategy  for  X  can  lead  to  y*(T)  =  0   so 
that  by  (64)  we  have 


S  (t=0)  =  (bv  -B)  -  bv   where   v 


=  0   for   y(T)  >  0, 
>  0   for   y(T)  =  0. 


(95) 


There  are  two  cases  to  be  considered: 

(a)  y(t=T)  >  0, 

(b)  y(t=T)  =  0. 
In  Case  (a) :   y(t=T)  >  0,   we  have  that 


ft* 


The  reader  should  recall  the  nonrestrictive  assumption  (65) . 

This  behavior  is  not  noted  in  either  [4]  (see  pp.  239-240)  or  [16] 
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S  (t=0)  =  bvy  -  B  >  0.  (96) 

By  continuity  of  y(t),   we  have  that   y(t)  >  0   for   t  e  (T-6,T]   where 

6  >  0   so  that  (55)  and  (96)  yield  that   S  (t)  >  0   for   0  <  x  <  T.   Hence, 

u 

(53)  yields  that  u*(x)  =  0  for  0  <  t  <  T,  and  a  backwards  integration 
of  the  state  equation  for  y(x)  yields  y(x)  >  0  for  0  ^  x  <  T.  Also, 
it  is  readily  seen  by  a  straightforward  continuity  argument  on   S  (t)   that 


u*(T)  =  0 


v*(t)  =  1 


for     t  e  [0,x1],  (97) 


where   x    denotes  the  "backwards  time"  of  the  first  switch  in  (extremal) 
tactics.   It  is  readily  seen  that   x(x)  >  0   for   x  e  [0,x  ).  The  dif- 
ferential equation  for  the  switching  functions  are  then  for  0  ^  x  <  x 

dS 
—  =  bA  with   S  (x=0)  =  bvv  -  B  >  0,       (98) 


dx  u  Y 

dS 


—  =  a{B  +  S  (x)}   with    S  (x=0)  =  avv  -  A  <  0,       (99) 


dx  u  v  X 

so  that  for   0  <  x  <  x 

S  (x)  =  bAx  +  bvv  -  B,  (100) 

u  i 

and 

Sv(T)  -  f  (Ax+vY)^  -  ||  v^  +  avx  -  A.  (101) 

The  switching  time   t  =  t.   is  determined  by   S  (x=x  )  =  0   and  hence 

V       1  V       V 


*!  •  \  '  JIM  +  %  <A-avx)-vY}.  d02) 

There   are    two    further   subcases    to   be   considered: 
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(1)  x(t=T-T1)    >    0, 

(2)  x(t=T-T1)    =   0. 


In   Subcase    (1):      x(t=T-T-.)    >   0,      continuity   arguments   on      x(t)      lead    to 

S    (x)    >   0      for      x    >    Tn .      Hence 
v  1 


u*(t)    =   0 
v*(t)    =  0 


for  t    e    (T    ,T],  (103) 


and      x(t)    >   0      for      0        t        T.      In   Subcase    (2): x(t=T-x..)    =  0,      we   have 

XX  X 

that      x(t)    =   0      for      x„    =   x      <    x    <    x      <   T     and      x(t)    >   0      for      x      <    x   <   T, 

£    1        e  e 

v 

Using  (62)  and  (63) ,  we  readily  obtain  for   x,  <  x  ^  x 

S  (t)  «  bAx  +  bvy  -  B  >  0,  (104) 

and 

S  (x)  -  0.  (105) 

Thus  we  have 


u*(t)  =  0 


v*(x)  =  1  -  — 
ay 


for     x  I    (t1,t^].  (106) 


X   X 
Since   x(x)  >  0   for   x  e  (x  ,x  +6),   arguments  similar  to  the  above  yield 

e   e 

V 

that   S  (x)  >  0   and   x(x)  >  0   for   x   <  x  <  T.   Hence 
v  e 


u*(x)  =  0 
v*(x)  =  0 


for     x  e    (xX,T].  (107) 

e 


In  Case  (b) :   y(t=T)  =  0,   there  are  two  further  subcases  to  be 
considered : 
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(1)  y(t)    =0      for      t      £    t    <   T     with      tY   <   T, 

e  e 

(2)  y(t=T)    =   0      but      y(t)    >0      for      T-6<t<T     where      6    >   0. 

Y  Y 

In  Subcase  (1):   y(t)  =  0   for   t   <  t  <   T  with   t   <  T,   it  is  readily 

seen  that  we  must  have 


U*(T)  -  1  -  ^ 

for     t  e:  [0,tY],  (108) 

e 


v*(t)  =  1 


so  that  a  result  analogous  to  (59)  ,  (62)  ,  and  (63)  yield  that  for 
Y    ^ 

0  <  T  <  T   <  Tn 

e    1 


Su(t)  =  0,  (109) 


and 


S  (t)  -  aBx  +  avv  -  A.  (110) 

v  X 

Y 
It  is  readily  seen  that  we  must  have   y(x)  >  0   for   t   <  t  <  T.   Thus 


for   x  e  (tY,tJ,  (HI) 

e   1 


,  u*(t)  -  0 
'  v*(t)  =  1 

where   x,   denotes  the  "backwards  time"  of  the  first  switch  in  (extremal) 

Y 
tactics  in  the  case  in  which   y(t)  =  0   for   0  <  T  <  T  ,   and 


u*(t)  =  0 
v*(t)  =  0 


for   t  €    (TlfT],  (U2) 


r^  Y 

by  (54)  and  (56),  since  y(t)  >  0   for   t  >  t±   >   Te.   Then  (62)  and  (63) 

Y       ^ 
become  for   t   <  t  <  t, 
e        1 
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=  bA 


with   S  (t=t  )  =  0, 
u    e 


dS 
i 

dT 

dSv  Y       Y 

- —  =  a{B+S  (t)}  with   S  (t=t  )  =  aBx   +  av  -  A  <  0, 

dx         u  v    e       e     X 


(113) 
(114) 


Y       ^ 
so  that  for   x  <  x  <  x„ 
e        1 


S  (t)  =  bA(x-T  ), 
u  e 


(115) 


and 


S  (t)  =  ^r   (t-tY)2  +  aB(T-TY)  +  aBTY  +  av  -  A. 
v       2      e  e       e     X 


(116) 


% 


The  switching  time   T  =  T-   is  determined  by   S  (t=t  )  =  0   and  hence 

v  1  v  v 


x      =   T      =   xY  +  ;Wb2   -    2bAB(xY   -    f^r]}    -   B),  (117) 

1  v  e        bALV  l    e         (      aB   J '  J 


Y        %. 
for      0   <    x      <    x'    = 
e  1 


A-av 


X  ^  ^    ,  YN 

— —   .      Thus,   we  have      t1    =   t1  (t    )  .       It    is   readily   seer 
aB  lie  A 

rA-av„i 


%  Y  Y 

that   t.   is  a  strictly  increasing  function  of   t   for  0  <  t   < 
1  e  e 


X 


aB 


since  we  have 


St, 


3t 


=  1  - 


/ 


rA-av 


(118) 


B2  -  2bAB{x* -^  J 


X 


aB 


3ti         y 

It  is  readily  seen  that  — —  >   0   for  x„  < 

.  Y  E 

3x 


A-av  \ 
x 


aB 


3t, 


,   that  — -  =   0   for 
9x 


Y 
x   = 
e 


A-av  ^ 
X 


aB 


8T1  Y 

,   and  that  — —  <  0   for   x   > 

8x  e 


A-avx, 


aB 


«v 


Hence,   x   has  a 


maximum  (which  we  denote  as   x')   for  x   = 

1  e 


A-av 


X 


aB 


It  seems  appropriate 


to  note   that 


and 


x      =   lim   x 
Y 

bA1 

x  +0 
e 

a.          ^, 

rA-avx- 

Tl    "    Tl    " 

aB 

2bAB 


A-av 


aB 


-B), 


(119) 


f  Y 

for        x 


rA-av 


aB 


(120) 
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It  should  be  noted  that 


*\*0   *v»    *>*■ 

X   <  T   <  T   - 
1      1      1 


A-av 


X 


{     aB  J  ' 


(121) 


for  0  <  x  < 
e 


A-av 


X 


aB 


As  we  discussed  for  Case  (II)  :   vY  <  -   and 

A      3. 


v     <  — ,   this  means  that  the  switching  time  for  v*(t)   depends  upon  force 
i    b 

levels,  since  it  depends  upon  whether  or  not  y(t)   is  driven  to  zero  before 
t  =  T.   Other  aspects  so  far  are  similar  to  this  previous  case. 

In  Subcase  (2):   y(t=T)  =  0  but  y(t)  >Q   for  T  -  6  <  t  <  T 
where   6  >  0,   it  is  clear  that  we  must  have   u*(t)  =  0   for  all 
t  e  (0,6..)  c  (0,6).   Thus,  we  have 


u*(t)  =  0 
v*(t)  =  1 


for   t  e  [0,t  ], 


(122) 


where   x   denotes  the  "backwards  time"  of  the  first  switch  in  (extremal) 
tactics  in  the  case  in  which  y(x=0)  =  0  but  y(x)  >  0   for  0  <  x  <  6. 
Considering  (122),  (64)  yields  that 


Su(t=0)  =  (bvy-B)  -  bv2  >  0, 


where 


0  <  v2  < 


bvy-B 


(123) 


(124) 


Using  (122)  in  the  integration  of  the  differential  equations  (62)  and  (63) 
for  the  switching  functions  with  the  boundary  conditions  (93)  and  (123), 
we  obtain  for  0  <  t  <  x.. 

S  (t)  =  bAx  +  bvy  -  B,  (125) 

and 

(126) 


ab 


ab 


8  (t)  =  f^Ax+(vY-v2)}2  -  f  (vy-v2)2  +  avY 


X 


It  should  be  noted  that  x(x),  y(x)  >  0   for  x  >  0, 
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The  switching  time   x   =  T,   is  determined  hy   S  (t=t  )  =  0   and  hence 

v    1  v    v 

TX  -  t  -  |)y<vY-v2)2  +  f (A-avx)  -  (vY-«2)},        (127) 


for   0  <  v   < 


bvy-B 


It  is  readily  seen  that 


u*(t)  -  0 
1  v*"(t)  =  0 


for   x  e  (t  ,T] 


(128) 


Thus,  we  have   x   =  x  (v  )   where   v   is  chosen  so  that   y(t=T)  =  0   but 


y(t)  >  0   for   T  -  6  <  t  <  T.   It  is  readily  seen  that   T   is  a  strictly 

(bvy-Bi 
increasing  function  of   v   for  0  <  v  < 


,   since  we  have 


8x 


3v 


i-lii- 


(vY-v2) 


Av 


■}    >0 


V*  +  t<A— X» 


(129) 


It   seems   appropriate    to   note    that 


i  "  *!  -  If/?7 


ab(A-avx)    "   VY 


for     v„  =  0, 


(130) 


and 


i<  o 
Tl 


1  {  /b  +2bAB 


bA 


A-av 


■av   A 


r     v, 


bvy-B 


(131) 


Thus,      t        is   a   strictly   increasing   function  of      v_,      and  we  have 


^  o 

T        <     T        <     T 
1    ■         1     "        1 


for        0   <   v      < 


bvy-B 


(132) 


It  should  also  be  noted  that   x    >  x    for  bv   >  B.   Thus,  from  (132)  we 
see  that  for  bv   >  B  more  than  one  extremal  can  lead  to  a  point   P   = 
(xf»yf)  =  (*f,0)   with   xf  >  0   on  the  terminal  manifold.   Thus,  in  Figure 
4  the  extremal  trajectories  labelled  as  (1),  (2),  and  (3)  all  lead  to  the 


terminal  manifold  with   yf  =  0 .   Let   P   =  (x-.,yn)   denote  a  point  in  the 
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initial  state  space  from  which  an  extremal  leads  to   P   =  (x  ,0)   with 
)v*(t)  =1   f°r   °  -  t  -  T  and  t1  <  T  <  t  °.   Furthermore,  let   C°   denote 
the  set  of  all  such  points,  i.e.   C  =  {P  |system  reaches  P  =  (x  ,0)   from 

lv*(t)  =  1   f°r  °  ~  t  ~  T  and   T1  -  T  -  t  °}.   From  the  backwards  synthesis 

procedure  for  the  extremal  strategic-variable  pair,  it  is  readily  seen  that 

the  set   C   lies  on  the  "backwards  extension"  of  the  extremal  which  originates 

(in  backwards  time)  from   (x.,,0)   with  xc   >  0 .   The  curve   C   is  shown  in 

r  r 

/  *  o  iu*(t)  =  0 

Figure  4  .   From  any  point  on  C    an  extremal  with  J  .,.,  ;    ,   for   0  ^  t  ^  T 

'  v*(t)  =  1 

leads  to  P  =  (x  ,0)   by  the  appropriate  choice  of  v  .   For  different  values 

of  P  ,   there  are  different  values  of  v  .   One  such  P   for  which  v   is 

dS 
—    %  o  v 

such  that   T..  <  x,  <  x    is  shown  in  Figure  4.   Since  - —  >  0,   we  must 
111  dT 


have  v*(t)  =  0   for  t  >  t   so  that  working  backwards  yields  that  the 

extremal  labelled  (2)  passes  through  P  .   Thus,  the  figure  suggests  (also 

look  at  extremals  emanating  from  P  )   that  there  are  several  ways  to  reach 

the  terminal  manifold  from  a  subset  of  the  state  space.   The  optimality  of 

extremal  trajectories  has  not  been  determined  for  this  special  case  of 

multiple  extremals.   We  will  report  our  results  to  date  with  the  final  deter- 

mination  for  the  complete  solution  to  (50)  in  Case  (12)  :   v  <  —  and 

a   a 

v__  >  —  being  deferred  to  the  future.   In  other  words,  the  solution  has  not 
I   b 

been  completely  determined  in  the  shaded  region  of  Figure  4. 

In  considering  Figure  4,  let  us  consider  the  case  in  which  we  have 

,  u*(t)  -  0 

<  for   0  <  t  <  T.  (133) 

[   v*(t)  =  v(t) 


The  reader  is  cautioned  that  Figure  4  represents  a  two-dimensional  (i.e.   t 
and  y)  illustration  of  the  projection  of  extremals  onto  a  plane  parallel  to 
the  plane   x  =  0   in  the  three  dimensional  state  space  (i.e.   t,  x,   and  y 
space).   For  notational  convenience,  we  are  not  being  precise  about  which 
entities  are  in  the  (t,x,y)-space  and  which  are  projections  onto  the  above 
noted  subspace.   Thus,   C°   denotes  both  a  figure  in   (t , x,y)-space  and  also 
its  projection  on   (t,y)-space.   Similar  remarks  apply  to  Figure  5. 
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Optimal   Y   Strategy 

Not  Determined  in  Shaded 

Region 


Transition 
Surface 


Optimal 
Trajectory 


time 


No t e :   St rategic  Variables 


X:   u 
Y:   v 


Figure  4.   Dependence  of  Y   Switching  Time   T-  =  T 

Upon  Force  Levels  for  hv   >  B   (not  drawn 
to  scale) . 
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Then  (133)  may  be  combined  with  the  state  equations  (50)  and  integrated  to 
yield 


and 


x(t)  =  x  +  rt  -  a 


y(t)  =  yQ  +  st  -  b 


0 
t 

0 


{1  -  v(s)}y(s)ds, 


x(s)ds 


(134) 


(135) 


The  above  may  be  combined  to  yield  the  following  (Volterra  integral  equation) 
for  y(t): 


y(t)    =  Yn  +   (s-bxjt  -  ^ft2  +  ab 


0 


0' 


ds. 


U-v(s2)}y(s2)ds2      (136) 


0  0 

When     v(s)    =  0      for     0   <   s   <   t,     we  obtain 

t     s. 


yV  °(t)  =  yQ  +  (s-bxQ)t  -  ^t2  +  ab 


dsl 
0     0 


y(s2)ds2,      (137) 


(138) 


and  when  v(s)  =  1   for  0  <  s  <  t,   we  obtain 

yV=1(t)  =  yO  +  (s-bxQ)t  -  ^t2. 

It  is  also  readily  shown  by  direct  integration  of  (50)  with  (133)  when 

v(s)  =  0   for  0  <  s  <  t   that 

-  s-kxn 

y    (t)  =  -(ay^-r)cosh^b  t  +  ( )sinh/ab  t  +  -. 

a    °  v^b" 

Considering  (137)  and  (138),  it  is  clear  that 

yV=°(t)  >  yV=1(t)   for   t  >  0,  (140) 


(139) 


provided  that  y(t)  >  0  and  that  strict  inequality  holds  for  a  finite 
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interval  of  time.   Hence,  whenever  an  (initial  )  point  lies  on   C    (such 
as   (B)  =  P   in  Figure  5),  if   Y   uses  the  extremal  strategy   v*(t)  =  0 
for   0  =  T  -  x,  <  t  <  T  -  x,   when  bv   >  B,   the  resulting  extremal  tra- 
jectory lies  (strictly)  above  that  corresponding  to   v* (t)  =  1   for 
0  <  t  <  T.   This  situation  is  shown  in  Figure  5  (which  is  expanded  detail 
of  Figure  4)  below.   Thus,  at  point   (B)  =  P    (which  lies  on   C  )   in 
Figure  5   Y   can  use  an  extremal  strategy  which  leads  either  in  the  direc- 
tion denoted  as  (3)  or  (4)  (and  results  in  extremals  denoted  as  (b)  and  (c)) 
Moreover,  if  the  trajectory  began  with   t  =  0   at,  for  example,  point  (A), 

then  the  extremal  strategy   v*(t)  =  0   for  0  <  t  <  T  -  x    must  have  been 

dS 
used,  since  if  extremal  (c)  is  chosen,   S  (t=t,  )  >  0   and  -r—  >   0,   and 

V     dSv 

if  extremal  (b)  is  chosen,   S  (1=1,)  =  0   and >  0.   Hence,   C   itself 

v    1  dx 

may  be  a  transition  surface  (depending  upon  the  choice  of   Y  when  C    is 
first  encountered).   Furthermore,  at  (A)   Y  can  choose  either  of  two 
extremal  directions  denoted  as  (1)  and  (2)  (and  resulting  in  an  extremal 
denoted  as  (a)  or  the  one  which  leads  to  (B)).   Thus,  the  problem  apparently 
has  multiple  extremals  (in  the  "triangular"  region  denoted  as   P  PQ) .   It 
appears  as  though  there  may  exist  extremal  trajectories  (such  as  the  one 
discussed  above)  at  various  points  along  which   Y  has  a  choice  of  alterna- 
tive extremal  strategies  (i.e.  a  bifurcation  occurs)  that  result  in  extremal 
trajectories  denoted  as  (a),  (b) ,  and  (c). 

It  has  not  been  determined  (by  direct  computation  of  the  criterion 
functional)  which  extremal  strategy  is  optimal  for  Y   in  this  special  set 


The  diagrams  of  battle  trajectories  in  this  report  are  sketched  for  a  forward 
evolution  of  the  battle  (e.g.  a  curve  of  y(t)   for  constant   xn)  •   Isaacs  [7] 
considers  a  backward  evolution  (e.g.  a  curve  of   y(x)   for  constant   xf  =  x(t=T)) 
Neither  approach  is  sufficient  to  determine  optimal  strategies.   One  must  either 
invoke  sufficient  conditions  for  optimality  or  use  the  solution  methodology 
referenced  in  [19]  (see  also  [18],  [21]).   We  hope  to  do  this  in  the  future. 
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Transition 
Surface 


Optimal 
Trajectory 


Figure  5.   Illustration  of  Multiple  Extremal  Strategies 


v*   for  Y  when  bv   >  B   (not  drawn  to  scale) . 
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of  circumstances.   However,  considering  well-known  sufficient  conditions 
of  global  optimality  based  on  convexity  for  optimal  control  problems,  the 
author  conjectures  that  all  extremal  strategies  are  optimal  for  Y  and 
lead  to  the  same  return.   If  this  is  true,  it  means  that  there  is  an 
"indifference"  set  in  Y's   optimal  strategy  space  such  that  he  can  vary 
the  survivability  of  his  aircraft  at  will  without  changing  the  value  of  his 

D 

criterion  functional.   It  should  be  noted  that  this  occurs  for   vv  >  — , 

i    b 

i.e.  the   Y  aircraft  (speaking  somewhat  imprecisely  now)  are  valued  greater 
than  the  return  obtained  from  gound  support  missions  at  the  end  of  the  war. 

Case  (III):   v   >  —  and   v  <  — . 

a  ■  ■  a        i   b 

A  B 

This  case  is  similar  to  Case  (12)  (vv  <  -  and  vv   >  — )   with  the 

X   a        id 

roles  of  X  and  Y  being  interchanged. 

Case  (112):   vw  >  -  and  vv  gff-. 
X a Y— — b 

By  Proposition  1  an  optimal  strategy  for   Y   can  lead  to   x*(T)  =  0 
so  that  by  (64)  we  have 


S  (t=0)  =  (av  -A)  -  av.   where   v 
u  XI  L 


=  0   for  x(T)  >  0, 
>  0   for   x(T)  =  0. 


There  are  two  cases  to  be  considered: 

(A)  x(t=T)  >  0, 

(B)  x(t=T)  =  0. 

Similarly,  by  Proposition  3  an  optimal  strategy  for   X  can  lead  to   y*(T)  =  0 
so  that  by  (64)  we  again  have  (95).   There  are  two  further  cases  to  be  con- 
sidered : 

(a)  y(t=T)  >  0, 

(b)  y(t-T)  =  0. 
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The  tedious  treatment  of  the  above  four  cases  (i.e.   (Aa) ,   (Ab) ,   (Ba) , 

and   (Bb)   where   (Aa)   denotes  that  we  have  x(t=T)  >  0   and  y(t=T)  >  0 

is  similar  to  that  done  in  Case  (12)  (v  <  -  and  vv  >  £)   and  therefore 

X   a        ib 

the  details  will  be  omitted.   It  may  be  shown  that  in  all  cases  (i.e.   (Aa) , 
(Ab),   (Ba),   and   (Bb))  we  must  have 


S  (t=0)  >  0  and   S  (t=0)  >  0, 
u  v 


(142) 


so    that,    for  example,   we  have     0   <   v     < 


av  -A 

x 


when  x*(t=T)  =  0.   Con- 


l   a  J 
sidering  (53),  (54),  (62),  (63),  and  (142),  arguments  similar  to  those 

used  above  readily  yield 

(  0       for   y(x)  >  0 


u*(t)  = 


1  -  r~     for  y(-r)  -  0 
bx 


for  all   t  e  [0,T].   (143) 


0 


for   x(t)  >  0 


v*(t)  = 

Extremals  are  unique  and  therefore  optimal. 


1  -  —  for  x(t)  =  0 
ay 


5.2.   Optimal  Air-War  Strategies. 

In  this  section  we  will  summarize  the  results  of  the  previous  section, 
In  summarizing  the  optimal  air-war  strategies  for  X  and  Y,   we  must 
consider  the  four  cases  (i.e.  (II),  (12),  (III),  and  (112))  delineated  above. 
Case  (II):   av  <  A  and  bv  <  B . 

X  x 

We  make  the  nonrestrictive  assumption  that 
A-av, 


rA-avxl 

< 

fB"bvY) 

bA  j 

I      aB  j 

(1)   If   T  < 


X 


aB    Tl 


U*(t,x,y)  =  1 
V*(t,x,y)  =  1 


for  0  <  t  <  T. 
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A-av 


(2)      If     — -r   <   T   <  -~{jA2+2aAB{ 


aB 


aB 


rB-bvy, 

bA~; 

- 

fA-avx] 

aB 

^                           J 

1  - 


av 


x}' 


/   U*(t,x,y)    -   1 
(  V*(t,x,y)    -     { 


for      x   >   0 


1  -  —     for     x  =  0 
ay 


for      0    <    t   <   T  - 


A-avx, 


aB 


and 


U*(t,x,y)    =   1 
V*(t,x,y)    =   1 


for      T  - 


A-av^ 


aB 


<   t   <   T, 


(3)      If      T   >  -~{ /A2+2aAB{ 


aB 


rB-bv   <, 
bA   j 

- 

rA-av   ^ 

A 

aB 

}  -  •',}  - 


T0, 


U*(t,x,y) 


V*(t,x,y)    = 


bx 


ay 


for      y   >   0 

for     y   =  0 


for      x   >   0 


for     x  =  0 


for      0   <    t   <    T  -    x    , 


where 


% 


(a)      x2  =  x2      for      x(t=T-T1)    >  0, 


<b>      %   =Tf  +^{/A+2aAB{ 


e        a 


B-bv^ 


bA 


-     T 


[}   -   A        for      x(t=T-x    )    =  0. 


X  X 

where      t        is    largest   value   of      x      such    that      x(t=T-x    )    =   0 

A-avx  x  B-bvy 

and      ~7r  =   Tl    "    Te    "    T2   =  — bA~    ' 

(c)      x„   =   x'       for      x(t=T-x,)    =   0     when      x      >    x', 
11  1  e  2 


also 

U*(t,x,y) 


=  1 


rA-av 


V*(t,x,y)    = 


0 

for 

x   >  0 

1  - 

r 
ay 

for 

x  -  0 

for      T-x<t<T- 


X 


aB      ' 


and 
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U*(t,x,y)  -  1 
V*(t,x,y)  =  1 


for  T  - 


A-av 


aB 


<  t  <  T. 


Case  (12):   av  <  A  and  bv„  >  B . 

A  Y 


(1)   If   T  < 


i{/v^+f(A-avx)  -  vy}  .  T 


U*(t,x,y)  =  | 
V*(t,x,y)  =  1 


for  y  >  0 


1  -  —  for   y  =  0     for  0  ^  t  5  T. 
bx 


(2) 


If  T *  iH  f\ +  f  (A"aV  -  vy1  ■  V 


U*(t,x 


V*(t 


0 

for 

y 

> 

0 

1  - 

s 
"  bx 

for 

y 

= 

0 

0 

for 

X 

> 

0 

1  - 

r 

for 

X 

= 

0 

for   0  <  t  <  T  -  t  , 


ay 


where 


% 


(a)  t1   =  x1      for  y(t=T)  >  0, 


(b 


)  ^^+^^(fA'aVx 


-  tY}  -  B}   for  y(t=T)  =  0, 


Y  Y 

where   t   is  largest  value  of   t   such  that  y(t=T-x  ) 

Y    %    A_aVX 

and  0  <  t   <  x'  =  r^  , 

el     aB 


=  0 


(c)   t,  =  xl      for  y(t=T)  =  0  when   x   >  x!  , 


A  final  determination  of  the  optimality  of  the  extremal  strategies  still 
remains  to  be  done  in  this  case.   Furthermore,  the  extremal  strategy  for 
Y  is  not  unique.   For  convenience  only  one  of  two  possibilities  is  given 
here . 
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and 


(    U*(t,x,y)    =     { 

)  l  1    -   £      for 

'    V*(t,x,y)    =    1 


for      y    >    0 

y  =  0 


% 


for      T   -    t      <    t    £>  T. 


Case  _( III)  : av     ^  A_    and      bv      <   B. 

(1)       If      T        Ij/v2+£(B-bv7-    vx}    -V 


U*(t,x,y)    7    1 


V*(t 


,x,y)    =     <  r 


for     x   >   0 


1 for      x   =  0 

ay 


(2)*      If     T>|f/v|  +  f(B-bvY)-vxl-x1, 


for      0        t        T, 


where 


*(t,x,y)    =     J 


for      y   >   0 


U* 


V*(t,x 


1  -  r—     for     y  =  0 
bx 


for      x   >   0 
r 


ay 


for     x  =  0 


(a)      T      =   t-       for      x(t=T)    >   0, 


for      0        t        T  -   x    , 


(b)      \    =   Te.  +  a^/A2+2aAB{ 


B-bv. 


bA 


-  t  }  -  A}   for   x(t=T)  =  0, 

e1     ' 

X  X 

where   t   is  largest  value  of   t   such  that   x(t=T=T  )  =  0 
e  e 

X   a,     B'bvY 
and  0  <  T  <  t'  =     *  , 
el     bA 


(c)   T-.  =  t'   for   x(t=T)  =  0  when   xX  >  t'  , 
11  el 


A  final  determination  of  the  optimality  of  the  extremal  strategies  still 
remains  to  be  done  in  this  case.   See  also  other  comments  in  footnote  for 
Case  (12). 
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and 


for   x   0 
for   z  =  0 


for   T  <  t.  <  t  <  T. 
r 

ay 


Case  (112):   av    A  and  bvw    B , 
X  Y 


U* ( t ,  x 


0       for  y  >  0 
y  -  0 


,  0       for 
v  1  -  : —   for 


bx 
0       for   x  >  0 


for   0  <  t  <  T, 


r*(t,x,y)  =  ( 

'  1  -  -—   fo 


r   x  -  0 

ay 


6 .   Some  Results  for  Case  of  Linearly-Decreasing-with-Time  Returns  from 
Ground  Support. 

One  criticism  of  the  constant  coefficient  version  (50)  of  the  general- 
ization (1)  of  the  tactical  air-war  game  is  that  the  criterion  functional 
does  not  adequately  reflect  timeliness  of  ground  support.   Intuitively,  we 
would  expect  that  support  of  the  ground  forces  is  "worth  more"  in  the  early 
stages  of  the  ground  war  than  later  on.   Thus,  we  are  led  to  consider  time- 
dependent  "returns"  from  ground  support  that  decrease  over  time.   Two  cases 
of  time-dependent  returns  from  ground  support  that  are  of  particular 
interest  are 

(1)  linearly-decreasing-with-time  returns  from  ground  support, 

(2)  exponentially-decreasing-with-time  returns  from  ground  support. 


Interest  in  these  cases  was  confirmed  in  informal  discussions  with  Maj  Ron 
Kronz,  USAF.   Such  facets  have  been  apparently  explored  in  numerous  runs 
of  TAC  CONTENDER. 
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Let   us    consider    the   case   of    (1)    in  which    Lite    LanchesLer    attrition-rate 
coefficients    for   counter-air   operations   are   constants    and    the    returns    from 
ground   support   are    time   dependent    (decreasing  over    time).      Thus,    we   will 
consider    the   case    in  which    the    following  hold: 

a(t)  ~   a  =   constant, 

b(t)  -  b  =   constant, 

A(t)  is  given    (decreasing)    function   of    time, 

B(t)  is  given    (decreasing)    function   of    time. 

For   notational    convenience  we  will   again   denote      a      as      a     and      b      as      b. 
It    is    then   convenient    to    re-state    the   problem  as    follows: 

t. 


[B(t)ux-A(t)vy]dt>, 


0 


maximize  minimize  {v  x(tf)-v   y(tp)  + 
U         V        X    t    Y    1 

with  stopping  rule:     t   -  T  =  0, 

dx 
subject    to:  —  =  r  -    (l-v)ay,  (144) 

(air-battle   dynamics) 

fjL.r  s  -  d-u)bx, 

with  initial  conditions   x(t=0)  =  x   and   y(t=0)  =  y_ , 
and 

x,y  Z   0   (State  Variable  Inequality  Constraints), 
0  <   u,v  <  1   (Strategic  Variable  Inequality  Constraints), 

where   a  and  b   are  given  constants  and  A(t)   and   B(t)   are  given 
f unc  tions . 

In  this  section  we  will  consider  the  case  in  which  both  A(t)   and 
B(t)   are  linearly  decreasing  functions  of  time  for   0  <  t  <  T.   The  situa- 
tion for   A(t)   is  shown  in  Figure  6.   We  have  then  that 
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and 


A(t)  =  AQ  -  t 


B(t)  =  BQ  -  t 


Ao~A^ 
0   T 


T    ' 


BQ-BT 

T 


(145) 


(146) 


where  AQ  >  AT  and   BQ  >  B  . 


A(t) 


A=0 


t=0 


t=T 


Figure  6.   Linearly-Decreasing-wi th-Time  Returns  from  Ground  Support  A(t) 
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We  will  now  review  the  necessary  conditions  of  optimality  that  we 
have  developed  for  (144)  in  Section  3  before  partially  synthesizing  the 
extremal  strategies  in  one  of  four  cases.   We  hope  to  give  a  more  thorough 
study  of  this  important  problem  in  the  future.   Let  us  recall  the  switching 
functions  in  backwards  time   t  =  T  -  t 


Su(t)  =  b(-q(x))  -  B(t), 


(147) 


Sv(t)  =  ap(-r)  -  A(t), 


where 


and 


A(t)  -  A(T-t)  =  AT  +  x 


(x)  =  B(T-t)  =  B?  +  t 


A0"AT 


B^~Bm 
0   T 


(148) 


(149) 


(15) 


Again,  extremal  strategies  are  given  by  (53)  and  (54)  for   x,y  >  0.   The 

differential  equations  for  the  switching  functions  are  given  by 

dB 

for      y   >   U, 


dS_u  ( 

dT  ) 


b{A(x)    +    (10v*)S    (t)}    - 


0 


for     y   >   0, 
for      y   =   0      (for   a    finite    interval    of    time), 


and 


^J 


dx 


rIA 

a{B(x)    +    (l-u*)S    (x)}    -  ~  for      x   >   0 , 

u  dx 


for      x  =  0      (for   a    finite    interval   of    time), 


(152) 


with    initial   conditions 


S    (x=0)    =    (bvv-B_)    -   bv,      and      S    (x=0)    =    (av   -A   )    -   av 
u  YTz  v  XT1 


(153) 


where      v        and      \>        must   satisfy    (60)    and    (61),    respectively.      It   is    also 


convenient    to  write    (151)    and    (i52)    as 
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and 


dS 
i 

dt 


b{AT+  t 


^Y^l    +    (1-v*)Sv(t)   - 


V^ 


for  y  >  0, 


(154) 


0      for  y  =  0   (for  a  finite  interval  of  time), 


dS 
> 

dr 


J 


a{BT+  t 


VBTl 

— —H  +   (1-u*)Su(t)}  - 


VAT 


for   x  >  0, 


to 


(155) 


for   x  =  0   (for  a  finite  interval  of  time) 


It  is  readily  shown  that  singular  solutions  are  impossible.   For  example, 
when   x,y  >  0   and   0  <  v*  <  1,   we  have 


d2S 
i 

dx^~ 


=  b 


W 


>  0, 


and  when  v*   is  constant  (i.e.  the  other  cases  in  which   v*  =  0   or   1), 
we  have 


d2S 


dx 


2~  =  b{v* 


V*f 


+  (l-v*)a[BT  +  t 


(VtI 

{      T      j 

+  (l-u*)S  (t)]} 
u 

d2S 


Observing  that   (l-u*)S  (x)  >  0,   it  is  clear  that   ,  n      >   0   for  all   x 

u  dx 

in  the  latter  case,  since  it  is  a  convex  combination  of  positive  quantities 

X        X 
On  a  constrained  subarc  on  which   x(t)  =  0   for   t   <  t  <  t   we 

e 

have  shown  the  following  to  be  necessary  conditions  for  optimality: 


v*(t)  =  1  - 


ay(t) 
r  <  ay(t) 
A(t) 


P(t)  = 


X      tx 

for   te  <  t  <  tv 


for   t   <  t  <  t. 
e        I 


for   t^,t  ::tj. 


(156) 
(157) 
(158) 


and 


B(t)u*  +  (l-u*)b(-q(t))  > 


0   T 


(159) 


Other  convenient  equivalent  forms  of  (159)  are 
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or 


a{B(t)    +    (l-u*)S    (t)}    2 
u 


maximum{B(t) ,    b(pq(t))} 


w 


A0  AT 


060) 


(161) 


It  is  to  be  noted  that  (159)  is  no  longer  automatically  satisfied  on  con- 
strained subarcs.   Similar  conditions  hold  on  a  constrained  subarc  on  which 
y(t)  =  0   for  a  finite  interval  of  time. 

Concerning  extremal  end  states,  we  have  developed  the  following. 


PROPOSITION  5: 

If  an  optimal  strategy  for  Y   is  to  result 

in  x*(T)  =  0,   then  it  is  necessary  that 

>   T 
v   £  —  . 

X    a 

PROPOSITION  6:   If   v   <  —  ,   then  an  optimal  strategy 
for  Y   leads  to   x*(T)  >  0. 


PROPOSITION  7: 

If  an  optimal  strategy  for  X  is  to  result 

in  y*(T)  =  0,   then  it  is  necessary  that 

• 

v   S  —  . 

Y    b 

PROPOSITION  8:   If   vv  <  —  ,   then  an  optimal  strategy 

i    b 

for  X   leads  to  y*(T)  >  0. 


In  synthesizing  an  extremal  strategic-variable  pair  there  are  four 
cases  to  be  considered: 


(I)  \  '  T 


(ID   vx  <  T 


(1)    vy  <  - 


(1)    vy  <  - 


(2)   vy  >  T 


(2)   vY  >  T 
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and 


dS 
v 

dT 


b{AT+  t 


^—l    +  (1-v*)Sv(t)   - 


B0"BT 


for  y  >  0, 


(154) 


0      for   y  =  0   (for  a  finite  interval  of  time), 


dS 
-\ 

dr 


J 


a{BT+  t 


VBTl 

~~-\    +   (1-u*)Su(t)}  - 


0   T 


for   x  >  0, 


to 


(155) 


for   x  =  0   (for  a  finite  interval  of  time) 


It  is  readily  shown  that  singular  solutions  are  impossible 
when   x,y  >  0   and  0  <  v*  <  1,   we  have 


For  example, 


d^S 


dx 


^  =  b 


'W 


>  0, 


and  when  v*   is  constant  (i.e.  the  other  cases  in  which   v*  =  0   or   1), 
we  have 


dzS 


dT 


Y~  -   b{v* 


A  -A 
°^-±]  +  (l-v*)a[BT  - 


VBT 


I 


+    (l-u*)S  (t)]} 
u 

d2S 


Observing  that   (l-u*)S  (t)  >  0,   it  is  clear  that   ■  ?   >  0   for  all   t 

u  dT 

in  the  latter  case,  since  it  is  a  convex  combination  of  positive  quantities 

X        X 
On  a  constrained  subarc  on  which   x(t)  =  0   for   t   <  t  <  t   we 

e 

have  shown  the  following  to  be  necessary  conditions  for  optimality: 


v*(t)  =  1  - 


ay(t) 
r  <  ay(t) 
A(t) 


P(t)  - 


for   t   <  t  <  t£> 


X        X 

for   t   <  t  <  t. 

e        x 


X        X 
for   t   <  t  <  t., 
e        I 


(156) 
(157) 
(158) 


and 


B(t)u*  +  (l-u*)b(-q(t))  >  - 

3 


rAQ-AT, 


(159) 


Other  convenient  equivalent  forms  of  (159)  are 


B-56 


a{B(t)  +  (l-u*)S  (t)} 
u 


rW 


(160) 


or 


maximum{ B(t) ,  b(pq(t))} 


A0  AT 


(161) 


It  is  to  be  noted  that  (159)  is  no  longer  automatically  satisfied  on  con- 
strained subarcs.   Similar  conditions  hold  on  a  constrained  subarc  on  which 
y(t)  =  0   for  a  finite  interval  of  time. 

Concerning  extremal  end  states,  we  have  developed  the  following. 


PROPOSITION  5:   If  an  optimal  strategy  for  Y   is  to  result 
in   x*(T)  =  0,   then  it  is  necessary  that 

v  ^   —  . 
X    a 


PROPOSITION  7: 

If  an  optimal  strategy  for  X  is  to  result 

in  y*(T)  =  0,   then  it  is  necessary  that 

v   £  —  . 

Y    b 

PROPOSITION  8:   If   v„  <  —  ,   then  an  optimal  strategy 

Y    b 

for  X   leads  to  y*(T)  >  0. 


In  synthesizing  an  extremal  strategic-variable  pair  there  are  four 


cases  to  be  considered: 

AT 

(I>   »s*T 


<">   vx  s  T 


(i)   vY<^ 


(i)   vY  <  T 


(2)   vy  *  ^ 


(2)   vY  >.  T 
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We  will  give  some  initial  results  in  the  first  of  the  above  cases. 

AT  BT 

In  Case  (11)  :   v„  <  - —  and   v„  <  -   ,   we  have  by  Propositions  6 
~~X a Y b — 

and  8  that   x*(T)  >  0   and   y*(T)  >  0   so  that  by  (J  5 3)  we  have 


S  (t=0)  =  bv  -  B  <  0   and   S  (t=0)  =  av„  -  A  <  0.         (162) 

U  I       J.  V  A       1 


By  the  usual  arguments,  i t  is  readily  seen  that 

u*(t)  =  1 


W*(t)  =  1 


for   t  e  [0,t  ], 


(163) 


and      x(x),y(x)    >   0      for      x    e  [0 ,  t    ) .      Using    (162)    and    (163),    the   differentia] 


equations    for    the   switching   functions   may  be    integrated    to   yield    for 
0   <    t    <    X 

b  fVM 


and 


VT>    ■   2 


VT>    "  f 


T      J 
B   -B 


T"1     + 


{b^ 


bo"bt1 

■V»tta»-  v 


VP  +   <aBT  - 


'A0"AT 


}x   +  avx  -   A    . 


(16A) 


(165) 


Let   t    denote  the  (only)  positive  root  of   S  (t=t  )  =  0   and  similarly 
u  u    u 

for   t  .   Then  we  have 
v 


■-    (T 


f      \1 
lA0-ATJ 

1 

"    b 

rVBT] 

A0-ATJ 

1+    /(tt 


f     ATl 

IA0-Atj 

1 
b 

fB0-BT] 

Ia0-atJ 

I2 


2T 


BT-bvy, 


A0"AT 


(166) 


and 


A0-ATl . 2 


VBT 


r  +  - 

1  a 


rAT-avx 


B0"BT 


(167) 


The  backwards  switching  time   t,   is  then  given  by 


T,  =  minimum  (x  ,x  )  <  T. 
1  u  v 


(168) 
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It  should  be  noted  that  in  order  for  the  switch  to  actually  occur  we  require 
that   t  ii   T.   Omitting  the  rather  tedious  details,  it  is  a  straightforward 
matter  to  prove  the  following  theorem: 


THEOREM  3:   In  order  that   t   e  [0,T)   for   B   >  bv  , 


(1)   it  is  necessary  that 


"*?]  <  k- 


(2)   it  is  necessary  and  sufficient  that 


(a) 


VV 


<bV 


and 


B  -bv 

o>)      f^lT}<T' 

A0  V 


A  similar  theorem  is  readily  proven  concerning  t    .   Thus,  we  see  that  in 
order  for  a  switch  in  extremal  strategies  from  (163)  to  occur  additional 
conditions  are  required.   This  should  be  contrasted  with  the  constant  coeffi- 
cient case  considered  in  Section  5  in  which  for  the  analogous  situation  (i.e. 


Case  (II)).   A  switch  in  extremal  strategies  from  (68)  was  guaranteed  if 

A-avx 
the  duration  of  the  battle  was  long  enough  (i.e.   T  >      -  ).   As  an  immediate 

3.  .D 

corollary    to   Theorem   3,    we   have 


COROLLARY  3.1: 

If 

'A   -A  ^ 

,°t  Tj  a  aBo  and 

'VETl 

,   T   J 

->bV 

then 

the  solution  to  (144)  with   A(t)   and 

B(t) 

given  by  (145)  and  (146)  is  given  by 

/U*ft)  =  1 

<            for  0  <  t  <  T. 

(v*(t)  =  1 
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It  should  be  noted  that  condition  (2b)  of  Theorem  3  may  be  written  as 

B(t=0)  -  bvy 

<  T,  (169) 

bA 

where 


a't 


T 
1 


A(t)dt 


Shortness  of  time  precludes  further  results  for  this  problem.   The  author 
hopes  to  give  further  details  in  the  future. 

7 .   Some  Results  for  Case  of  Exponentially-Decreasing-with-Time  Returns  from 
Ground  Support. 

In  this  section  we  will  briefly  consider  the  case  in  which  both  A(t) 

and   B(t)   are  exponentially  decreasing  functions  of  time  for  0  <  t  <  T. 

We  have  then  that 

A(t)  =  AQ  e"at,  (170) 

and 

B(t)  =  BQ  e"3t'  (171) 

where   a,  6  >  0. 

We  will  now  review  the  review  the  necessary  conditions  of  optimality 
that  we  have  developed  for  (144)  in  Section  (3)  before  partially  synthesizing 
the  extremal  strategies  in  one  of  four  cases.   Again,  we  hope  to  give  a  more 
thorough  study  of  this  important  problem  in  the  future.   As  before,  the 
extremal  strategies  are  given  by  (53)  and  (54)  for   x,y  >  0.   The  differen- 
tial equations  for  the  switching  functions  are  given  by 


B--60 


dS  ,  b(l-v*)Sv(T)    +    (bAQe   aT)oaT    -    (3BQe    PT)eBl      for      y   >    0, 

— y  =    < 
dx  i 

0     for   y  =  0   (for  a  finite  interval  of  time), 


and 


(172) 


,_     ,  a(l-u*)S  (t)  +  (aBne  BT)e6t  -  (aA_e  aT)eaT   for   x  >  0, 
db     (  u         0  0 

T^  =  < 
dx     | 

0     for   x  =  0   (for  a  finite  interval  of  time), 
with  initial  conditions 


(173) 


Su(x=0)  =  (bvY-BQe  BT)  -  bv2  and   Sv(x=0)  =  Cavx-AQe  aT)  -  a\)^,   (174) 

where   v,   and   v„   must  satisfy  (60)  and  (61),  respectively. 

X        X 
On  a  constrained  subarc  on  which   x(t)  =  0   for   t  ^    t  ?  t  ,   we 

e 

have  shown  that  (156)  through  (158)  are  necessary  conditions  for  optimality 

and  also 

A 
B(t)u*  +  (l-u*)b(-q(t))  >  — -  e  at.  (175) 

a. 

Other  convenient  equivalent  forms  of  (175)  are 


or 


a(B(t)  +  (l-u*)S  (t))  >  ctA  P~at, 
u         0 


maximum  (b (t) ,b (-q (t) ) }  >  e 


(176) 


(177) 


As  in  the  previous  case,  it  is  to  be  noted  that  (175)  is  no  longer  auto- 
matically satisfied  on  constrained  subarcs.   Similar  conditions  hold  on  a 
constrained  subarc  on  which   y(t)  =  0   for  a  finite  interval  of  time. 

Concerning  extremal  end  states,  we  have  developed  the  following. 


PROPOSITION  9: 

If  an  optimal  strategy  for  Y   is  to  result  in 

x*(T)  =  0,   then  it  is  necessary  that 

A  e-aT 
V 

v   > 

X    a 
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Aoe 

PROPOSITION  10:   If   v   <  — ,   then  an  optimal  strategy  for 

X        3. 

strategy  for  Y   leads  to   x*(T)  >  0. 


PROPOSITION  11:   If  an  optimal  strategy  for   X   is  to  result  in 

y*(T)  =  0,   then  it  is  necessary  that 

DQe 

v   > , 

Y      b 


PROPOSITION  12: 

If 

-bt 
<  Boe 

VY        ™b~  '   ' 

then  an   optimal   strategy 

for 

X      leads    to 

y*(T)    >   0. 

In  synthesizing  an  extremal  strategic-variable  pair  there  are  four 
cases  to  be  considered: 


V 


-BT 


Be 

(1)   vY<-\ 


-BT 


(2)   vy  > 


V 


-BT 


V 
(ID   vx, 


-aT 


a 

(1) 

VY 

< 

b 

BT 

(2) 

VY 

> 

V" 

b 

BT 

We  will  now  give  some  initial  results  in  the  first  of  the  above  cases 


In  Case  (II):   v  < 

X 


V 


-aT 


v"ST 

and  bw  < ,   we  have  by  Proposi- 

Y b  — 


tions  10  and  12  that   x*(T)  >  0   and   y*(T)  >  0   so  that  by  (174)  we  have 


-BT 


-aT 


S  (t=0)  =  bvy  -  BQe  p   <  0   and   Sv(x=0)  =  avx  -  AQe    <  0 


(178) 


By  the  usual  arguments,  it  is  readily  seen  that 

,u*(t)  -  1 
<v*(t)  =  1 


for   t  e  [0,1^^], 


(179) 
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and   x(x),y(x)  >  0   for   t  e  [0,t  ).   Using  (178)  and  (179),  the  differential 
equations  for  the  switching  functions  may  be  integrated  to  yield  for 
0  <  t  <  T 

-aT 


1 


and 


S  (t)  = 
u 


S  (t) 

V 


ai    ,   -6T.  St    ( 
e    -  (BQe    )e   +  b  vy  - 


V 


St    .        -aT.  ax  , 
e    -  (A  e    )e   +  a 


-3T 


vx  " 


(180) 


(181) 


Let   x   denote  the  smallest  positive  root  of   S  (t=t  )  =  0   and  similarly 
u  u    u 

L"or   t  .   Then  the  backwards  switching  time   t    is  given  by 


t.  =  maximum  (t  ,t  )  ?  T 

1  U   V 


(182) 


We  will  now  give  some  results  for  the  following  two  cases: 

(1)  o..-  3, 

(2)  6  >  a. 

In  Case  (1):   a  =  3,   in  order  for   t    to  exist  such  that   S  (x=T  )  =  0 
u  u    u 

we  must  have 


and  then 


bAQ  >  3BQ, 


±  b(A0-3e6Tvy) 

Tu  =  a  in    *  bA.  -  SB, 


(183) 


(184) 


0     0 
Omitting  the  details  for  now,  the  following  theorem  may  be  proven: 
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THEOREM 

4 :   In  order 

that   T   e  [0,'D   for   a  =  3   and 

BQe 

bvy, 

(1) 

dR 
it  is  necessary  that   —  (t=0)  <  bA(t=0)  (where 

A(t)   and   B(t)   are  given  by  (170)  and  (171)) 

(2) 

it  is  necessary  and  sufficient  that 

(a) 

||(t=0)  <  bA(t=0), 

and 

B(t=0)  -  bvv 

(b) 

1             J  <    T, 

bA 

where 

T 

Wj 

A(t)dt. 

0 

A  similar  theorem  concerning  T    may  be  proven.   The  reader  should  note 
that  Theorems  3  and  4  are  basically  identical.   Thus,  there  is  great  simi- 
larity (at  least  for  the  results  so  far  obtained)  between  the  cases  of 
J inearly-decreasing-wi th-t imo  returns  from  ground  support  and  exponentially- 
decreasing  ones.   As  above,  we  see  that  in  order  for  a  switch  in  extremal 
strategies  from  (179)  to  occur  additional  conditions  in  addition  to  the  war 
being  of  sufficient  duration  are  required.   This  should  be  contrasted  with 
the  constant  coefficient  case  (i.e.  Case  (II))  considered  in  Section  5. 
As  an  immediate  corollary  to  Theorem  4,  we  have 


COROLLARY 

4 

.1: 

For  the  case  in  which   a  =  8,   if 

~(t=0)  >  aB(t=0)   and  ~(t=0)  >  bA(t=0) 
dt                     dt 

then  the  solution  to  (144)  with  A(t)   and 

B(t)   given  by  (170)  and  (171)  is  given  by 

.  u*(t)  =  1 

<                                   for  0  <  t  <  T. 

W*(t)  -  1 
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In  Case  (2) :   3  >  a,   the  following  theorem  may  be  shown  to  hold 


THEOREM  5: 


In  order  that   t   g  [0,T)   for   3  >  a   and 

n 

B(t=T)  >  bv  ,   it  is  sufficient  that 
(a)   |f(t=0)  <  bA(t=0), 


and 


(b)   { 


B(t=0)  -  bv 


}  <  T. 


bA 


Shortness  of  time  precludes  further  resu] ts  for  this  problem.   The  author 
hopes  to  give  further  details  in  the  future. 


8 •   Discussion. 

In  this  appendix  we  have  applied  the  theory  of  differential  games 
(including  the.  theory  of  state  variable  inequality  constraints  developed 
in  Appendix  A)  to  the  study  of  optimal  air-war  strategies.   We  have  con- 
sidered a  generalization  of  the  tactical  air-war  game  studied  by  Isaacs 
[7]  and  others  [1],  [4],  [6],  [16],  [24].   We  saw  that  when  one  considers 
salvage  values  for  aircraft,  time-dependent  returns  from  ground  support, 
and  time-dependent  Lanchester  attrition-rate  coefficients,  previous  solution 
methodology  for  the  tactical  air-war  game  can  result  in  non-optimal  strate- 
gies.  For  example,  during  the  early  phases  of  the  air  war,  it  may  not  be 
optimal  to  annihilate  enemy  aircraft  whenever  possible.   We  also  saw  that 
when  there  is  a  residual  value  for  aircraft  at  the  "end"  of  the  war,  optimal 
strategies  may  be  appreciably  different  than  those  for  the  special  case 
previously  considered. 

We  developed  a  (fairly)  complete  solution  to  the  generalization  of 
the  tactical  air-war  game  for  the  case  of  constant  coefficients.   Some  work 
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still  remains  to  be  done  in  one  special  case.   We  propose  this  to  ONR  as  a 

future  research  task.   We  also  partially  developed  extremal  strategies  in 

two  cases  of  time-varying  (decreasing)  returns  from  ground  support  (both 

linearly-decreasing  returns  and  also  exponentially-decreasing  ones).   In 

these  cases  we  have  shown  that  optimal  strategies  for  both  sides  may  be  to 

always  concentrate  on  supporting  the  ground  forces  regardless  of  how  long 

the  conflict  lasts.   For  the  case  in  which   av  <  A(t=T)   and  bv   <  B(t=T), 

X  i 

we  showed  that  an  additional  condition  had  to  be  satisfied  (namely, 

-r-(t=0)  <  bA(t=0))   in  order  for  it  to  be  possible  for  a  switch  in  X's 
at 

optimal  strategy  to  occur.   (It  should  be  noted  that  this  latter  condition 
is  always  satisfied  for  the  case  of  constant  returns  from  ground  support.) 

The  results  of  this  appendix  are  not  only  of  intrinsic  interest  but 
are  also  useful  for  checking  the  computational  adequacy  of  Lagrange  dynamic 
programming,  which  has  been  proposed  as  a  compational  method  for  time-sequential 
combat  games  [13],  [14].   It  should  be  pointed  out  that  there  may  be  severe 
computational  difficulties  in  using  Lagrange  dynamic  programming  when  the 
solution  to  a  time-sequential  combat  game  has  multiple  extremals  (as  is 
frequently  the  case  (see  [19])). 

We  think  that  an  understanding  of  the  parameters  which  affect  optimal 
air-war  strategies  is  essential.   In  particular,  the  effects  on  optimal 
strategies  of  considering  salvage  values  for  aircraft,  time-dependent  returns 
for  ground  support,  and  time-dependent  Lanchester  attrition-rate  coefficients 
are  not  entirely  understood  at  this  time.   We  feel  that  an  analytic  approach 
like  that  employed  here  is  very  useful  for  discovering  cause-effect  relation- 
ships between  optimal  strategies  and  modelling  assumptions.   We  have  laid 
the  foundations  for  future  research  in  this  area  and  recommend  such  future 
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study    to   ONR.      Of   particular   importance,    is    to   determine  whether   or   not 
the    hasic   model    is    adequate.      A  basic    shortcoming   of    the    model     (1)     is    that 
it   does    not   evaluate   air-war   tactics  within    the   context   of    ground-war 
objectives.      The   model   suboptimizes .       In   the   next   appendix  we   briefly 
consider    a   model    that   does    evaluate   air-war    tactics   within    the   context 
of   ground-war   objectives. 


B-67 


REFERENCES 


[1]  L.  Berkovitz  and  M.  Dresher,  "A  Game  Theory  Analysis  of  Tactical  Air 
War,"  Opns.  Res.  7,  599-620  (1959). 

[2]  L.  Berkovitz  and  M.  Dresher,  "Allocation  of  Two  Types  of  Aircraft  in 
Tactical  Air  War:  A  Game  Theoretic  Analysis,"  Opns.  Res.  8,  694-706 
(1960). 

[3]   A.  Bryson  and  Y.  C.  Ho,  Applied  Optimal  Control,  Blaisdell  Publishing 
Co.,  Waltham,  Massachusetts  (3  969). 

[4]   A.  Friedman,  Differential  Games,  Wiley-Interscience ,  New  York  (1971). 

[5]   D.  Fulkerson  and  S.  Johnson,  "A  Tactical  Air  Game,"  Opns.  Res.  5, 
704-712  (1957). 

[6]   L.  Giamboni,  A.  Mengel,  and  R.  Dishington,  "Simplified  Model  of  a 

Symmetric  Tactical  Air  War,"  The  RAND  Corporation,  RM-711,  August  1951 

[7]   R.  Isaacs,  Differential.  Games,  John  Wiley  and  Sons,  Inc.,  New  York 
(1965). 

[8]   D.  Jacobson,  M.  Lele,  and  J.  Speyer ,  "New  Necessary  Conditions  of 

Optimality  for  Control  Problems  with  State-Variable  Inequality  Con- 
straints," J.  Math.  Anal.  Appl .  35,  255-284  (1971). 

[9]   Y.  Kawara,  "An  Allocation  Problem  of  Fire  Support  in  Combat  as  a 
Differential  Game,"  Opns.  Res.  21,  942-951  (1973). 

[10]   F.  W.  Lanchester,  Aircraft  in  Warfare;   The  Dawn  of  the  Fourth  Arm, 
Constable  and  Co.,  London  (1916). 

[11]   P.  Morse  and  G.  Kimball,  Methods  of  Operations  Research,  M.I.T.  Press, 
Cambridge,  Massachusetts  (1951). 

[12]   E.  Paxson,  "War  Gaming,"  The  RAND  Corporation,  RM-3489-PR,  February 
1963. 

[13]  G.  Pugh,  "Theory  of  Measures  of  Effectiveness  for  General-Purpose 
Military  Forces:  Part  II.  Lagrange  Dynamic  Programming  in  Time- 
Sequential  Combat  Games,"  Opns.  Res.  21,  886-906  (1973). 

[14]   G.  Pugh  and  J.  Mayberry,  "Analysis  of  General-Purpose  Force  Procure- 
ment," Lambda  Corporation,  Paper  59,  August  19  71. 

[15]   G.  Pugh  and  J.  Mayberry,  "Theory  of  Measures  of  Effectiveness  for 

General-Purpose  Military  Forces:   Part  I.   A  Zero-Sum  Payoff  Appro- 
priate for  Evaluating  Combat  Strategies,"  Opns.  Res.  21,  86  7-885 
(1973). 


B-68 


[16]   S.  Sternberg,  "Development  of  Optimal  Allocation  Strategies  in 

Heterogeneous  Lanches Ler-Type  Processes,"  Report  No.  SRL  2147  TR  71-1 
(U),  Systems  Research  Laboratory,  The  University  of  Michigan,  Ann  Arbor, 
Michigan,  June  1971. 

[171  J-  Taylor,  "Comments  on  a  Multiplier  Condition  for  Problems  with 
State  Variable  Inequality  Constraints,"  IEEE  Trans,  on  Automatic 
Control,  Vol .  AC-17,  74  3-744  (19  72). 

[18]   .1.  Taylor,  "Application  of  Differential  Games  to  Problems  of  Military 
Conflict:   Tactical  Allocation  Problems  -  Part  II,"  Naval  Postgraduate 
School  Technical  Report  No.  NPS55Tw721 J 1A,  November  1972. 

|J9]   J.  Taylor,  "Survey  on  the  Optimal  Control  of  Lanches ter-Type  Attrition 

Processes,"  Naval  Postgraduate  School  Technical  Report  No.  NPS55Tw74031 , 
March  19  74. 

|20J  J.  Taylor,  "On  the  Solution  to  Lanchester-Type  Equations  of  'Modern 
Warfare'  with  Variable  Coefficients,"  Opns .  _Res . ,  to  appear. 

[21]   J.  Taylor,  "The  Isbell-Marlow  Problem  Revisited,"  submitted  to  Naval 
Res.  Log.  Quart . 

[22]  J.  Taylor,  "On  Some  Multiplier  Conditions  for  Control  Problems  with 
State  Variable  Inequality  Constraints,"  submitted  to  IEEE  Trans .  on 
Automatic  Control . 

[23]   C.  Thomas  and  W.  Deeiner,  "The  Role  of  Operational  Gaming  in  Operations 
Research,"  Pjjns^_Res_._ J5_,  1-27  (1957). 

[24  1   USAF  Assistant  Chief  of  Staff,  Studies  and  Analysis,  "Methodology  for 
Use  in  Measuring  the  Effectiveness  of  General  Purpose  Forces,  SABER 
GRAND  (ALPHA),"  March  19  71. 


APPENDIX  E:   Some  Problems  for  Future  Study. 

1 .   Introduction . 

In  this  appendix  we  will  briefly  describe  several  important  problems 
for  possible  future  research.   These  problems  arc  all  extensions  of  work 
reported  botli  here  in  this  report  and  elsewhere  [9],  [10]. 

As  we  have  pointed  out  recently  [11],  when  we  began  our  research,  the 
state-of-the-art  of  deterministic  control  theory  (at  least  that  for  applica- 
tions to  operations  research  (see  [12]))  was  not  sufficient  to  allow  routine 
solution  of  even  the  simplest  Lanchester-type  optimal  control  problem.   Our 
research  approach  has  been  to  consider  a  sequence  of  idealized  models  of 
increasing  mathematical  complexity  and  also  increasing  operational,  realism. 

At  this  point  in  time,  most  of  the  simplest  Lanchester-type  tactical 
allocation  structures  have  been  fairly  thoroughly  studied.   Some  theoretical 
results  in  optimal  control  theory  have  resulted  from  this  examination  [12], 
[13].   To  solve  the  problems  at  hand  that  we  propose,  we  would  build  upon 
the  knowledge  gained  by  solving  numerous  particular  examples. 

The  problems  that  we  will  elaborate  upon  below  may  be  referred  to  as 

(1)  fire  support  for  several  ground  units, 

(2)  incorporation  of  logistics  considerations, 

(3)  effect  of  the  form  of  the  criterion  functional  in  combat 
optimization  problems. 

All  the  above  problems  are  quite  important.   The  first  is  important  because 

it  is  a  closer  approximation  to  the  type  of  decision  that  a  commander  must 

make  in  allocating  supporting  fires  to  aid  various  friendly  units  than  we 

have  previously  considered.   The  second  is  important  because  it  incorporates 

logistics  considerations  into  Lanchester-type  equations  of  warfare.   As  far 

as  Navy  missions  are  concerned,  this  is  very  important,  since  sea-based 
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supply  and  re-supp]y  is  a  principle  Navy  combat  support  mission.   The  third 
is  important  because  it  provides  more  insight  into  the  relationship  between 
the  structure  of  optimal  strategies  and  the  nature  of  (system)  objectives 
(here  nonlinear) . 

As  we  first  pointed  out  in  [11],  for  the  purposes  of  military  analysis, 
it  is  convenient  to  consider  that  there  are  three  essential  parts  of  any 
dynamic  combat  optimization  problem: 

(a)  the  decision  criteria  (for  both  combatants), 

(b)  the  model  of  conflict  termination  conditions  (and/or  unit 
breakpoints) , 

(c)  the  model  of  combat  dynamics. 

Our  proposed  problems  (1)  and  (2)  above  fall  into  the  category  of  research 
on  the  effects  on  the  structure  of  optimal  strategies  of  aspect  (c),  whereas 
(3)  falls  into  the  category  of  research  on  (a).   It  is  this  central  analysis 
aspect  of  the  combatants'  decision  criteria  that  the  author  believes  is  the 
weakest  link  in  the  chain  of  assumptions  in  current  analyses.   Pugh  and 
Mayberry  [4]  have  recently  proposed  methodology  for  this  important  topic 
of  measures  of  effectiveness  in  dynamic  combat  optimization  problems. 
However,  they  do  not  explore  the  consequences  of  various  functional  forms 
of  the  criterion  functional.   This  is  the  objective  of  studying  problem  (3). 

Finally,  it  should  be  pointed  out  that  all  the  problems  considered 
below  are  stated  in  their  simplest  form.   This  is  because  of  our  research 
objective  of  studying  solution  properties,  an  objective  whose  attainment  is 
facilitated  by  the  availability  of  complete,  explicit  analytic  solutions. 
More  realistic  versions  of  these  problems  are  readily  formulated.   However, 
mathematical  tractability  is  sacrificed  when  this  is  done.   Moreover,  our 
results  for  such  idealized  problems  provides  guidance  for  developing  numeri- 
cal solution  algorithms  for  more  realistic  campaign  models. 


E-3 


2 •   Fire  Support  of  Several  Ground  Units. 

Consider  combat  between  heterogeneous   X  and   Y   ground  forces 
(infantry).   Contact  exists  between  these  forces  along  the  Forward  Edge 
of  the  Battle  Area  (FEBA) .   The   X   ground  forces  are  composed  of  two  units, 
denoted  as   X.   and   X  ,   and  similarly  for  the   Y   ground  forces.   It  is 
assumed  that  the.   X   forces  are  only  in  contact  (i.e.  combat)  with  the 
Y    forces  and  similarly  for  X  .   In  each  of  the  two  "one-on-one"  battles 
both  combatants  undergo  a  "square-law"  attrition  process  (see  [6])  from 
enemy  (small  arms)  fire. 

The  problem  facing  the   X  commander  is  to  determine  the  "best"  dis- 
tribution of  the  fire  of  his  artillery,  denoted  as   U,   over  the  enemy  units, 
i.e.   Y   and  Y  .   The  artillery   U   fires  at  a  constant  rate  into  the 
(constant)  area  of  an  enemy  unit  without  feedback  as  to  the  destructiveness 

of  this  fire.   This  situation  is  shown  in  Figure  1.   X's   decision  criterion 
;'c  ** 

is  the  net  worth   of  survivors  (with  a  linear  utility  for  survivors   ).   For 
the  purposes  of  this  report,  discussion  of  the  stopping  rule  (i.e.  battle 
termination  model)  is  not  essential  and  is  omitted. 

Mathematically,  the  problem  may  be  stated  as 

T        T 
maximize     {v  x(T)  -  w  y(T)}, 

dXl 
subject  to:   -r—  =  -a^y,  ,  O) 


dx 


■  ~a0y, 


dt      2'2' 


Pugh  and  Mayberry  [4]  suggest  using  the  ratio  of  aggregated  force  value 
For  our  purposes  here  the  exact  form  of  the  criterion  functional  is  not 
essential. 

See  [2]  for  methodology  for  the  development  of  these  linear  utilities. 
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Infantry 


Artillery 
(Naval  Guns) 


Figure  1.   Diagram  for  Fire  Support  of  Several  Ground  Units 
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and 


dyi 

IT  =  "bixi  ~  *cJv 

dy2 

dT~  =  _b2x2  "  (1-*)^y2' 

with  initial  conditions 

x.(t=0)  =  x°   and   y.(t=0)  =  y°   for   i  =  1,2, 
1  1         1         J  x 

x-,»^2,yl,y2   ~   °  (State   Variable    Inequality   Constraints), 

0    <   <J>    <   1  (Contro]    Variable    Inequality   Constraints), 


where 


x.(t)   [for   i  =  1,2]   is  the  number  of   X.   infantry  at  time   t, 
similarly  for  y.(t), 

a.   [for   i  =  1,2]   is  the  constant  (Lanchester)  attrition-rate 

coefficient  (the  rate  at  which  one   Y.   unit  can  destroy  X. ) , 

i  i 

similarly  for  b . , 

J  l 

c.   [for   i  -  1,2]   is  the  constant  (Lanchester)  attrition-rate 

coefficient  for  the  effectiveness  (area  fire)  of   U  artillery 
against  Y. , 

vT  =  (vrv2), 

v.   is  the  value  of  one  surviving  X.   unit, 
l  l 

similarly  for  w. 

To  be  precise,  in  the  above  formulation  we  have 

-a.y.   for  x.  >  0, 
dx.    I    xJ x  x 


for   x.  =  0, 


and 
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-b1xl  -  (j^y,   for   yn  >  0, 


U 


dYl       ul"l    ^l'l   LUL   '1 


0  for   y   -  0. 

Moreover,  we  would  begin  by  considering  the  case  in  which   x  ,   x  ,   y  , 
and   y°  are  such  that   xx (T) ,x  (T) ,y  (T) ,y  (T)  >  0. 

It  should  be  clear  that  the  above  problem  (1)  is  important  because 
it  considers  a  basic  aJ location  problem  for  artillery  fire.   With  minor 
modifications  the  model  (1)  may  be  extended  to  other  cases  of  interest: 
(a)  let  b   =  b.;  =  0   for  a  prescribed  duration  battle  attack  scenario 
(see  Appendix  D),  (b)  for  an  amphibious  assault  one  adds  replacements  for 
the   X.   and   K»   forces. 

Instead  of  being  called  "artillery,"  the  fire  support  units   U  may 
be  called  "Naval  guns."   Of  particular  relevance  to  Navy  problems  in  the 
determination  of  the  optimal  allocation  of  Naval  gunfire  in  amphibious 
assaults . 

Finally,  it  seems  appropriate  to  discuss  a  technical  difficulty  of 
solving  the  above  problem  (1).   As  are  all  Lanches ter-type  fire  distribution 
problems,  the  above  problem  (1)  is  a  singluar  problem  of  optimal  control, 
since  ~~j   =  0   (see  [7]),  where   H  denotes  the  Hamiltonian.   The  problem 
becomes  particularly  difficult  when  a  singular  solution  exists  in  x  -  p 
space  (see  [1]).   A  preliminary  investigation  has  shown  that  this  is  the 
case  for  the  problem  at  hand.   It  should  be  pointed  out  that  we  have  encoun- 
tered such  difficulty  previously  in  our  research  reported  here  (see  Appendix 
D). 
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3 .   A  Lanchester-Type  Optimal  Control  Problem  with  Logistics  Considerations. 

In  this  section  we  will  describe  a  simple  model  which,  nevertheless, 
can  generate  some  insights  into  tactics  used  by  General  George  S.  Patton, 
Jr.,  in  World  War  II.   Consider  combat  between   X  and  Y   forces.   Part  of 
the   X   forces  (denoted  as   X  )   can  be  kept  in  reserve  and  consequently  do 
not  consume  supplies  as  rapidly  as  the  combatant  forces  (denoted  as   X..  )   do. 
It  is  assumed  that  the  effectiveness  of   X..   against   Y  depends  upon  the 
amount  of  supplies  (denoted  as   S)   that  they  have.   For  the  problem  under 
consideration,  this  effectiveness  will  be  represented  by  a  Lanchester  attri- 
tion-rate coefficient   b1  =  b..(S).   It  seems  reasonable  to  hypothesize  that 
b1  (S)   is  a  concave  function  of   S  with  certain  other  appropriate  properties 
Further  research  should  be  done  on  this  important  topic.   For  convenience  in 
our  initial  investigation,  we  would  assume  that  b    is  constant  but  that 
S  >  0.   We  further  assume  that  due  to  "pipeline"  capacity  there  is  an  upper 
limit  to  the  rate  at  which  supplies  can  be  replenished. 

The  decision  variable  under  the  control  of  the   X  commander  is  the 
rate  of  reinforcing  (or  withdrawing  for   u(t)   negative)  the   X   forces. 
This  situation  is  shown  in  Figure  4.   The  objective  of  the   X  commander 
is  to  maximize  the  net  worth  of  survivors  (considering  linear  utilities). 

In  mathematical  terms,  the  problem  may  be  stated  as  follows. 

maximize  {px  (T)  +  qx„(T)  -  ry(T)}   with   T  unspecified, 
u(t) 

dxl 
subject  to:  -r—  =  -ay  +  u,  (2) 


dt 

-ay 

-t- 

u, 

dx_ 
dt 

-u, 

dt 

-bx^ 

i 

dS 
dt 

-CX1 

+ 

P, 
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Figure  2.   Diagram  of  Supply-Constrained  Combat  Problem. 


with  initiai  conditions 


x.(t=0)  =  x.   for   i  =  1,2,   y(t=0)  =  yQ ,   S(t=0)  -  SQ, 


and 


x  ,x  ,y,S  >  0     (State  Variable  Inequality  Constraints), 

-W  <  u  <  R      (Control  Variable  Inequality  Constraints), 
and  with  the  stopping  rule 

(a)  x1(T)  =  0, 

(b)  y(T)  =  0, 
or          (c)     S  <  0, 


where 


c   denotes  the  rate  of  consumption  of  supplies   S  by  one   X 
comb a tan  t , 

R,W  >  0, 
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P  denotes  the  "pipeline"  capacity, 

S   denotes  the  supply  level  of  the   X.   forces, 

u(t)   denotes  the  rate  of  reinforcement  of   X,   forces  by  the 
X   forces  (negative  quantity  denotes  withdrawal), 

and  other  notation  is  similar  to  that  used  in  the  statement  of  problem  (1). 

A  preliminary  examination  of  problem  (2)  has  yielded  an  interesting 
result:   if  supplies  are  to  become  a  constraining  factor  for  the   X   forces 
later  in  combat,  then  the  optimal  tactic  is  to  "overcommit"  forces  early  in 
the  campaign  (i.e.  forces  are  being  withdrawn  at  the  moment  that  the  supply 
constraint  becomes  active).   Of  particular  mathematical  difficulty  is  the 
presence  of  a  second  order  SVIC  (i.e.   S  >  0)  (see  [8])  in  this  problem. 

The  only  previous  work  considering  logistics  aspects  apparently  is 
by  Moglewer  and  C.  Payne  [3].   Our  above  problem  (2)  may  be  extended  to 
yield  insights  for  policy  planning  regarding  "pipeline  capacity"  (logistics 
capacity),  pipeline  protection,  pipeline  interdiction,  etc. 

4 .   Form  of  the  Criterion  Functional  in  Time-Sequential  Combat  Optimization 
Problems . 

In  all  our  past  research,  we  have  with  one  exception  (see  Appendix  D 

of  this  report)  always  considered  a  linear  utility  for  survivors   in  the 

criterion  functional.   It  is  of  interest  to  examine  how  the  valuation  of 

survivors  affects  the  structure  of  optimal  strategies.   It  seems  appropriate 

to  begin  such  an  investigation  by  considering  the  simplest  problem  possible. 

Thus,  we  will  consider  a  version  of  the  "Tactical  Air-War  Campaign"  (see 

[10]  and  Appendix  B  of  this  report). 


■k 

See  [2]  for  methodology  for  the . development  of  such  a  linear  utility 
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Mathematically,  the  problem  may  be  stated  as  follows 

[x(l-T)-y]dt}, 


lf 


and 


maximize  {Q(x(t  )  ,y  (t  )  )  + 
<f>(t)         ±  f     J 

TV  Q 


with  stopping  rule:    tf  -  T  =  0 


,  .  dx 

subject    to:  —  =   r, 

(air-battle   dynamics) 

^   =  s   -   <|>bx, 


with  initial  conditions 


x(t=0)  =  xQ   and   y(t=0)  =  y  , 


x,y  >  0       (State  Variable  Inequality  Constraints), 
0  <  <f>  <   1    (Control  Variable  Inequality  Constraints), 


where 


x(t)   and   y(t)   denote  the  numbers  of   X  and   Y  aircraft, 
respectively,  at  time   t, 

b   denotes  a  Lanchester  attrition-rate  coefficient  (the  rate 
at  which  one   X  aircraft  shoots  down  Y  aircraft), 

r   and  s   denote  replacement  rates, 

Y(t)   denotes  the  fraction  (at  time   t)  of  total   X  aircraft 
which  fly  counter-air  missions  and  hence  shoot  down   Y 
aircraft, 

and    Q(x(tf ) ,y (t f) )   denotes  the  (salvage)  value  of   X  and  Y 

aircraft  at  the  end  of  the  planning  horizon. 


It  is  of  interest  to  examine  the  dependence  of  the  structure  of  the 
optimal  allocation  policy  for  X  aircraft  upon  the  natiare  of   G(xf,yf), 
where   xf   denotes   x(t=tf),   etc.   It  seems  reasonable  on  military  grounds 
to  require  that 
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T~~  >   0      and     ^-  <   o      for   all  xc,y£    >   0. 

3xf                           3yf  f,Jf 

Furthermore,    one   might   postulate    either   of    the  following   functional    forms 
for     Q(xf,yf): 

(a)           Q(xf,yf)    -   F(xf)    -  G(yf ) , 


or 


F(xf) 

(b)  Q(xf)yf)    = 


f"f         G(yf)     * 

It  is  of  interest  to  study  cases  in  which   F   (and/or   G)   is  a 

(A)  concave  function, 

or     (B)  convex  function, 

or     (C)  quasi-concave  function, 

or     (D)  quasi-convex  function. 

To  consider  a  concrete  example,  if   G(y  )   were  a  concave  function,  it  might 
look  like  that  shown  in  Figure  3.   One  possibility  for  an  analytic  representa- 
tion of  the  function  shown  in  Figure  3  is 

30     -ay 
G(yf)  -  -~   (1-e   r).  (4) 

After  the  dependence  of  the  structure  of  the  optimal  allocation 
policy  upon  the  nature  of  the  terminal  return  G(xf ,yf)   has  been  studied 
for  the  above  problem  (3)  (preliminary  analysis  indicates  that  this  problem 
is  analytically  tractable),  it  would  seem  appropriate  to  consider  a  problem 
like  the  Isbell-Marlow  fire  distribution  problem  (see  [5],  [6],  [8],  or  [10]), 
which  is  at  the  next  level  of  analytical  complexity.   Such  a  research  program 


Pugh  and  Mayberry  [4]  have  suggested  using  the  ratio  of  aggregated  force 
values . 


rV 


(it 


Total   value 
of      Y 
survivors 


G(yf) 
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Number  of   Y   survivors,   y 


Figure  3.   Decreasing  Marginal  Value  for  Survivors 


would  lead  to  a  better  understanding  of  the  effects  upon  (optimal)  military 
decision  making  of  the  valuation  of  (system)  objectives.   This  in  turn  would 
result  in  a  better  understanding  of  the  optimization  of  combat  dynamics. 
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